these 14 derivations will be built: /nix/store/bnh71h8x2990rkgv4y40x5i9c72p3lka-python3.14-nixbot-effects.drv /nix/store/3v7zbvl3706j0qmcbihabln7gxrxb0gh-python3.14-nixbot.drv /nix/store/siixi34191n7255gs7jbn87nab3kbzsh-unit-nixbot.service.drv /nix/store/9rprbqavwn9p32xfc5sdayql86q5anz6-system-units.drv /nix/store/6cflxb7y3nlsdlac32ldh4f6x0lqar15-etc.drv /nix/store/3pgqzwh6bh5ijdrqvgfxym0zhh0qppm0-activate.drv /nix/store/0xjx18pxmwji8l1jd9a4mzyq16d6k3fp-nixos-system-gitlab-test.drv /nix/store/01yf2kk8lss9qhr7zlv8icnjq7vz4bx5-closure-info.drv /nix/store/2c7ck1chlb8n9fc4119p3qbiwsl8lyik-closure-info.drv /nix/store/2hjb6hgc1c88plfqyv0j1272n0ggyfhw-run-nixos-vm.drv /nix/store/fpn2znbcbhznd2vn00pi6xdfk3lbbr6j-nixos-vm.drv /nix/store/5y0pm5hjicamh0szpw9w6d1lc5k6y5ck-driverConfiguration.json.drv /nix/store/y76065gj9cxxk9qx85727kwzx3agw1g6-nixos-test-driver-nixbot-gitlab.drv /nix/store/b15y2ywbr33hlkc9hj9aqgh5gkhg601a-vm-test-run-nixbot-gitlab.drv building '/nix/store/siixi34191n7255gs7jbn87nab3kbzsh-unit-nixbot.service.drv' unit-nixbot.service> tribuchet: building on eliza unit-nixbot.service> structuredAttrs is enabled building '/nix/store/9rprbqavwn9p32xfc5sdayql86q5anz6-system-units.drv' system-units> tribuchet: building on eliza building '/nix/store/6cflxb7y3nlsdlac32ldh4f6x0lqar15-etc.drv' etc> tribuchet: building on eliza building '/nix/store/3pgqzwh6bh5ijdrqvgfxym0zhh0qppm0-activate.drv' activate> tribuchet: building on eliza building '/nix/store/0xjx18pxmwji8l1jd9a4mzyq16d6k3fp-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> tribuchet: building on eliza nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/01yf2kk8lss9qhr7zlv8icnjq7vz4bx5-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/2c7ck1chlb8n9fc4119p3qbiwsl8lyik-closure-info.drv' closure-info> tribuchet: building on eliza closure-info> structuredAttrs is enabled building '/nix/store/2hjb6hgc1c88plfqyv0j1272n0ggyfhw-run-nixos-vm.drv' run-nixos-vm> tribuchet: building on eliza building '/nix/store/fpn2znbcbhznd2vn00pi6xdfk3lbbr6j-nixos-vm.drv' nixos-vm> tribuchet: building on eliza building '/nix/store/5y0pm5hjicamh0szpw9w6d1lc5k6y5ck-driverConfiguration.json.drv' driverConfiguration.json> tribuchet: building on eliza driverConfiguration.json> structuredAttrs is enabled building '/nix/store/y76065gj9cxxk9qx85727kwzx3agw1g6-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> tribuchet: building on eliza nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! building '/nix/store/b15y2ywbr33hlkc9hj9aqgh5gkhg601a-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> tribuchet: building on eliza vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.01 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 46) vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.Wyl8S2qMOV', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.4 (6-Mar-2025) vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 76bf97d0-0d0a-442f-a3a7-ef255ef0c21a vm-test-run-nixbot-gitlab> (finished: start all VMs, in 199.28 seconds) vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] KASLR enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Machine model: linux,dummy-virt vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] efi: UEFI not found. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA32 empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: probing for conduit method from DT. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Trusted OS migration not required vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-v4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-BHB vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] alternatives: applying boot alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/alj08f6mkbvgig08wd02jcynxd5q27gq-nixos-system-gitlab-test/init regInfo=/nix/store/zygf9w8nga14yi5gnl905das149xcxz4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/zygf9w8nga14yi5gnl905das149xcxz4-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 256 SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000033] arm-pv: using stolen time PV vm-test-run-nixbot-gitlab> gitlab # [ 0.000409] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.000640] Console: colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.000648] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.000841] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) vm-test-run-nixbot-gitlab> gitlab # [ 0.000849] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.000939] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.001063] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.001066] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.001501] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.001716] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.001831] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.003038] cacheinfo: Unable to detect cache hierarchy for CPU 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.003790] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.003795] rcu: Max phase no-delay instances is 1000. vm-test-run-nixbot-gitlab> gitlab # [ 0.003958] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.005126] fsl-mc MSI: its@8080000 domain created vm-test-run-nixbot-gitlab> gitlab # [ 0.005223] EFI services will not be available. vm-test-run-nixbot-gitlab> gitlab # [ 0.005420] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.007391] Detected PIPT I-cache on CPU1 vm-test-run-nixbot-gitlab> gitlab # [ 0.007549] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007623] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 vm-test-run-nixbot-gitlab> gitlab # [ 0.007700] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.009096] Detected PIPT I-cache on CPU2 vm-test-run-nixbot-gitlab> gitlab # [ 0.009163] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 vm-test-run-nixbot-gitlab> gitlab # [ 0.009237] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 vm-test-run-nixbot-gitlab> gitlab # [ 0.009312] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.011055] Detected PIPT I-cache on CPU3 vm-test-run-nixbot-gitlab> gitlab # [ 0.011138] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 vm-test-run-nixbot-gitlab> gitlab # [ 0.011214] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 vm-test-run-nixbot-gitlab> gitlab # [ 0.011292] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] vm-test-run-nixbot-gitlab> gitlab # [ 0.012341] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.012371] SMP: Total of 4 processors activated. vm-test-run-nixbot-gitlab> gitlab # [ 0.012373] CPU: All CPU(s) started at EL1 vm-test-run-nixbot-gitlab> gitlab # [ 0.012384] CPU features: detected: Branch Target Identification vm-test-run-nixbot-gitlab> gitlab # [ 0.012389] CPU features: detected: ARMv8.4 Translation Table Level vm-test-run-nixbot-gitlab> gitlab # [ 0.012392] CPU features: detected: Instruction cache invalidation not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.012395] CPU features: detected: Data cache clean to the PoU not required for I/D coherence vm-test-run-nixbot-gitlab> gitlab # [ 0.012399] CPU features: detected: Common not Private translations vm-test-run-nixbot-gitlab> gitlab # [ 0.012402] CPU features: detected: CRC32 instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.012405] CPU features: detected: Data cache clean to Point of Deep Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.012408] CPU features: detected: Data cache clean to Point of Persistence vm-test-run-nixbot-gitlab> gitlab # [ 0.012411] CPU features: detected: Data independent timing control (DIT) vm-test-run-nixbot-gitlab> gitlab # [ 0.012414] CPU features: detected: E0PD vm-test-run-nixbot-gitlab> gitlab # [ 0.012417] CPU features: detected: Enhanced Counter Virtualization vm-test-run-nixbot-gitlab> gitlab # [ 0.012420] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) vm-test-run-nixbot-gitlab> gitlab # [ 0.012423] CPU features: detected: Enhanced Virtualization Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.012426] CPU features: detected: Fine Grained Traps vm-test-run-nixbot-gitlab> gitlab # [ 0.012430] CPU features: detected: Generic authentication (architected QARMA5 algorithm) vm-test-run-nixbot-gitlab> gitlab # [ 0.012435] CPU features: detected: RCpc load-acquire (LDAPR) vm-test-run-nixbot-gitlab> gitlab # [ 0.012438] CPU features: detected: LSE atomic instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.012441] CPU features: detected: Privileged Access Never vm-test-run-nixbot-gitlab> gitlab # [ 0.012444] CPU features: detected: PMUv3 vm-test-run-nixbot-gitlab> gitlab # [ 0.012447] CPU features: detected: RAS Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.012449] CPU features: detected: RASv1p1 Extension Support vm-test-run-nixbot-gitlab> gitlab # [ 0.012452] CPU features: detected: Random Number Generator vm-test-run-nixbot-gitlab> gitlab # [ 0.012455] CPU features: detected: Speculation barrier (SB) vm-test-run-nixbot-gitlab> gitlab # [ 0.012457] CPU features: detected: Stage-2 Force Write-Back vm-test-run-nixbot-gitlab> gitlab # [ 0.012460] CPU features: detected: TLB range maintenance instructions vm-test-run-nixbot-gitlab> gitlab # [ 0.012465] CPU features: detected: Speculative Store Bypassing Safe (SSBS) vm-test-run-nixbot-gitlab> gitlab # [ 0.012570] alternatives: applying system-wide alternatives vm-test-run-nixbot-gitlab> gitlab # [ 0.015650] CPU features: detected: BBM Level 2 without TLB conflict abort vm-test-run-nixbot-gitlab> gitlab # [ 0.015888] Memory: 5964036K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.017161] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.026059] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.026733] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.026977] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL vm-test-run-nixbot-gitlab> gitlab # [ 0.026981] 0 pages in range for non-PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.026983] 508288 pages in range for PLT usage vm-test-run-nixbot-gitlab> gitlab # [ 0.027153] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.028040] DMI not present or invalid. vm-test-run-nixbot-gitlab> gitlab # [ 0.031401] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.033970] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.034242] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.036238] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.036267] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.036486] audit: type=2000 audit(0.036:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.037034] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.037036] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.037040] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.037042] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.037045] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.037102] cpuidle: using governor ladder vm-test-run-nixbot-gitlab> gitlab # [ 0.037119] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.037322] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. vm-test-run-nixbot-gitlab> gitlab # [ 0.037456] ASID allocator initialised with 65536 entries vm-test-run-nixbot-gitlab> gitlab # [ 0.038644] Serial: AMBA PL011 UART driver vm-test-run-nixbot-gitlab> gitlab # [ 0.044547] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 vm-test-run-nixbot-gitlab> gitlab # [ 0.044796] printk: console [ttyAMA0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.051516] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.051536] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.051541] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.051546] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.051550] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.051554] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.051559] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.051563] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.067682] fbcon: Taking over console vm-test-run-nixbot-gitlab> gitlab # [ 0.067717] ACPI: Interpreter disabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.069694] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 0.069704] iommu: DMA domain TLB invalidation policy: strict mode vm-test-run-nixbot-gitlab> gitlab # [ 0.077671] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.078341] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 0.078404] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 0.078431] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 0.078865] pps_core: LinuxPPS API ver. 1 registered vm-test-run-nixbot-gitlab> gitlab # [ 0.078871] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti vm-test-run-nixbot-gitlab> gitlab # [ 0.078900] PTP clock support registered vm-test-run-nixbot-gitlab> gitlab # [ 0.078956] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.079376] scmi_core: SCMI protocol bus registered vm-test-run-nixbot-gitlab> gitlab # [ 0.080009] FPGA manager framework vm-test-run-nixbot-gitlab> gitlab # [ 0.081951] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.088477] clocksource: Switched to clocksource arch_sys_counter vm-test-run-nixbot-gitlab> gitlab # [ 0.089223] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.089259] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 0.089857] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.090795] pnp: PnP ACPI: disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.099449] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.101349] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.144697] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.144769] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.144837] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.145253] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.145962] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 0.146100] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.146176] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.146324] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.146577] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.146647] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.146669] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.147180] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 0.156773] kvm [1]: HYP mode not available vm-test-run-nixbot-gitlab> gitlab # [ 0.213394] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 0.213709] workingset: timestamp_bits=42 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 0.214394] squashfs: version 4.0 (2009/01/31) Phillip Lougher vm-test-run-nixbot-gitlab> gitlab # [ 0.214507] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 0.226561] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 0.226574] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 0.226772] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) vm-test-run-nixbot-gitlab> gitlab # [ 0.226947] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 0.226952] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 0.241139] pl061_gpio 9030000.pl061: PL061 GPIO chip registered vm-test-run-nixbot-gitlab> gitlab # [ 0.244548] ledtrig-cpu: registered to indicate activity on CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.245354] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.245370] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.245402] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.245409] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.245445] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits vm-test-run-nixbot-gitlab> gitlab # [ 0.245479] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.245634] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.245653] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.245668] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.245672] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.245675] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.245787] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.246346] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.246835] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.246853] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.246887] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.246900] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.247637] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.247865] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.247877] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.247906] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.248524] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.248766] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.248778] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.248807] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.249415] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.249642] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.249654] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.249683] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.250671] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.250899] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.250912] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.250946] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.251525] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.251750] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] vm-test-run-nixbot-gitlab> gitlab # [ 0.251763] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.251796] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.252365] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.252604] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.252617] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.252645] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.252657] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.253233] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.253462] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.253490] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.254276] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.254505] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.254533] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.255281] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.255507] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.255785] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.256112] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.256125] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.256154] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.256740] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.256968] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] vm-test-run-nixbot-gitlab> gitlab # [ 0.256980] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.257014] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.257721] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257729] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257733] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257781] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257827] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257880] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257931] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.257982] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258033] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258081] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258131] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258179] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258257] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258304] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258323] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258347] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258370] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258393] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258417] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258437] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258456] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258480] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258505] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258526] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258546] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258564] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258582] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258601] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258621] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258639] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258658] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258677] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258696] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258716] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned vm-test-run-nixbot-gitlab> gitlab # [ 0.258739] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.258744] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.258747] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.260245] pci 0000:00:0a.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.273973] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.277190] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.280409] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.284097] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.289751] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.292546] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.295464] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.298069] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.306510] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) vm-test-run-nixbot-gitlab> gitlab # [ 0.308810] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.312717] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) vm-test-run-nixbot-gitlab> gitlab # [ 0.322426] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.324903] msm_serial: driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.325125] SuperH (H)SCI(F) driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.325179] STM32 USART driver initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.370895] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.371744] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.374501] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.378462] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 0.381845] virtio_blk virtio5: [vdb] 12395592 512-byte logical blocks (6.35 GB/5.91 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 0.388081] megasas: 07.734.00.00-rc1 vm-test-run-nixbot-gitlab> gitlab # [ 0.389010] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.392217] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.392252] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.395848] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.395915] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.398458] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.398489] Intel/Sharp Extended Query Table at 0x0031 vm-test-run-nixbot-gitlab> gitlab # [ 0.408704] Using buffer write method vm-test-run-nixbot-gitlab> gitlab # [ 0.408737] Concatenating MTD devices: vm-test-run-nixbot-gitlab> gitlab # [ 0.408740] (0): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.408745] (1): "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.408747] into device "0.flash" vm-test-run-nixbot-gitlab> gitlab # [ 0.753818] Freeing initrd memory: 27184K vm-test-run-nixbot-gitlab> gitlab # [ 0.766580] tun: Universal TUN/TAP device driver, 1.6 vm-test-run-nixbot-gitlab> gitlab # [ 0.772460] thunder_xcv, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.772627] thunder_bgx, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.772657] nicpf, ver 1.0 vm-test-run-nixbot-gitlab> gitlab # [ 0.773536] e1000: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.773544] e1000: Copyright (c) 1999-2006 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.773583] e1000e: Intel(R) PRO/1000 Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.773593] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.773629] igb: Intel(R) Gigabit Ethernet Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.773634] igb: Copyright (c) 2007-2014 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.773663] igbvf: Intel(R) Gigabit Virtual Function Network Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.773669] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. vm-test-run-nixbot-gitlab> gitlab # [ 0.773882] sky2: driver version 1.30 vm-test-run-nixbot-gitlab> gitlab # [ 0.777455] usbcore: registered new interface driver usb-storage vm-test-run-nixbot-gitlab> gitlab # [ 0.777670] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 0.777691] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 0.778684] hv_vmbus: registering driver hyperv_keyboard vm-test-run-nixbot-gitlab> gitlab # [ 0.779475] ehci-pci 0000:00:0a.0: EHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 0.779536] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.779904] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 vm-test-run-nixbot-gitlab> gitlab # [ 0.780266] rtc-pl031 9010000.pl031: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 0.780313] rtc-pl031 9010000.pl031: setting system clock to 2026-11-07T10:58:35 UTC (1794049115) vm-test-run-nixbot-gitlab> gitlab # [ 0.780909] i2c_dev: i2c /dev entries driver vm-test-run-nixbot-gitlab> gitlab # [ 0.785492] sdhci: Secure Digital Host Controller Interface driver vm-test-run-nixbot-gitlab> gitlab # [ 0.785501] sdhci: Copyright(c) Pierre Ossman vm-test-run-nixbot-gitlab> gitlab # [ 0.785930] Synopsys Designware Multimedia Card Interface Driver vm-test-run-nixbot-gitlab> gitlab # [ 0.786556] sdhci-pltfm: SDHCI platform and OF driver helper vm-test-run-nixbot-gitlab> gitlab # [ 0.788543] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 vm-test-run-nixbot-gitlab> gitlab # [ 0.789271] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 0.789301] hub 1-0:1.0: 6 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 0.789537] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 0.790005] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 0.790012] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 0.798110] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available vm-test-run-nixbot-gitlab> gitlab # [ 0.814769] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 0.815080] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.816955] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.816976] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 0.817078] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.817297] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 0.820611] Key type dns_resolver registered vm-test-run-nixbot-gitlab> gitlab # [ 0.832210] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.832616] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 0.850594] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 0.850818] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 0.850824] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 0.850984] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 0.851005] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 0.851037] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 0.852720] input: gpio-keys as /devices/platform/gpio-keys/input/input0 vm-test-run-nixbot-gitlab> gitlab # [ 0.877104] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 0.877141] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 0.881436] Freeing unused kernel memory: 4736K vm-test-run-nixbot-gitlab> gitlab # [ 0.881665] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 0.917950] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 1.044537] usb 1-1: new high-speed USB device number 2 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.195165] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 vm-test-run-nixbot-gitlab> gitlab # [ 1.252403] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 1.252455] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 1.252574] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 1.252583] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 1.253187] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 1.253350] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 1.329048] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.460593] usb 1-2: new high-speed USB device number 3 using ehci-pci vm-test-run-nixbot-gitlab> gitlab # [ 1.488601] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 1.559298] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 1.575741] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 1.576027] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 1.576062] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 1.576076] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 1.576102] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.576119] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.576137] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 1.576155] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.576355] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.576702] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 1.576865] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 1.577034] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.577137] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 1.577158] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 1.579019] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 1.581070] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 1.581156] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 1.583308] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 1.599673] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 1.602034] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 1.602191] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 1.604446] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 1.608080] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 1.611393] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 1.613605] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 vm-test-run-nixbot-gitlab> gitlab # [ 1.618420] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 1.619072] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 1.619560] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 1.622743] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 1.645605] systemd-journald[95]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 1.650729] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 1.652285] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.652461] [drm] features: -virgl +edid -resource_blob -host_visible vm-test-run-nixbot-gitlab> gitlab # [ 1.652505] [drm] features: -context_init vm-test-run-nixbot-gitlab> gitlab # [ 1.653892] [drm] number of scanouts: 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.653910] [drm] number of cap sets: 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.660844] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 1.660872] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 1.660898] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.662586] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 1.663998] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 1.667242] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 1.674957] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 1.699149] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 1.704759] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 1.708260] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 1.711198] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.714054] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 1.721037] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 1.724899] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 1.734559] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 1.730136] systemd-modules-load[96]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 1.733479] systemd-modules-load[96]: Module 'virtio_balloon' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.734691] systemd-modules-load[96]: Module 'virtio_console' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.735859] systemd-modules-load[96]: Module 'virtio_rng' is built in vm-test-run-nixbot-gitlab> gitlab # [ 1.737572] systemd-modules-load[96]: Inserted module 'dm_mod' vm-test-run-nixbot-gitlab> gitlab # [ 1.738652] systemd-modules-load[96]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 1.739957] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 1.758220] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 1.760776] systemd-udevd[109]: Using default interface naming scheme 'v261'. vm-test-run-nixbot-gitlab> gitlab # [ 1.780734] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 1.808603] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 1.875456] systemd-vconsole-setup[132]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 1.879069] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.250397] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 2.251935] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 2.253392] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 2.420101] (udev-worker)[142]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.440621] (udev-worker)[140]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 2.444355] (udev-worker)[140]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.447712] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.462827] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.464159] systemd[1]: Reached target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 2.465509] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.515992] systemd-fsck[154]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 2.521668] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.605225] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 2.642591] EXT4-fs (vda): mounted filesystem 76bf97d0-0d0a-442f-a3a7-ef255ef0c21a r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 2.642396] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 2.643689] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 2.645630] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 2.687002] systemd-sysroot-fstab-check[162]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 2.688934] systemd[1]: Reload requested from client PID 162 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 2.691001] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 2.811092] systemd[1]: Reloading finished in 120 ms. vm-test-run-nixbot-gitlab> gitlab # [ 2.868491] systemd-sysroot-fstab-check[162]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 2.871293] systemd-sysroot-fstab-check[162]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 2.873993] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.876005] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 2.877571] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 2.879925] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 2.913006] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.915946] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 3.106039] (udev-worker)[123]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 3.110489] (udev-worker)[123]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory vm-test-run-nixbot-gitlab> gitlab # [ 3.119842] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.121293] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.122101] systemd[1]: Stopping Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.123333] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.130866] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.132182] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.142563] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.191079] systemd-vconsole-setup[185]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 3.193928] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.608126] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.610333] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.615519] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 3.657842] systemd[1]: Mounting /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 3.666285] systemd[1]: Mounting /sysroot/tmp/xchg...[ 3.671505] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 3.686068] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.689534] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.690864] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 3.692110] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 3.697074] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.699776] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 3.731082] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 3.732869] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.607043] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 4.682325] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 4.683811] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.685805] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 4.688391] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 4.725789] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.728275] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.747893] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.749303] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.750826] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 4.799752] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.801435] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 4.802599] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 4.803900] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.805178] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.806891] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 4.808738] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.809877] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.811068] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 4.812436] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 4.815826] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.817166] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.819620] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.822222] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.828329] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.829726] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 4.831414] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.833109] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.836341] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 4.837917] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.839585] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 4.841401] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.843332] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.845375] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.846879] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 4.848223] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.849390] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.850786] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.852365] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 4.853568] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 4.855125] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.856609] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 4.857670] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.859071] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.860391] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.861848] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.862978] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.864383] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 4.865493] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.866687] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.868077] systemd[1]: systemd-udevd.service: Consumed 2.503s CPU time over 3.144s wall clock time, 31.6M memory peak. vm-test-run-nixbot-gitlab> gitlab # [ 4.869733] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.870856] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.871663] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.872575] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.873770] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.874766] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.876171] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.877254] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.878347] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.941577] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.943459] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.944722] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.945880] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 5.115431] initrd-nixos-activation-start[223]: booting system configuration /nix/store/alj08f6mkbvgig08wd02jcynxd5q27gq-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 5.175497] initrd-nixos-activation-start[223]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 5.574238] initrd-nixos-activation-start[247]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 5.742891] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 5.744775] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.746194] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.785973] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 5.945218] systemd-journald[95]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 6.642126] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 6.642217] systemd[1]: Detected virtualization qemu. vm-test-run-nixbot-gitlab> gitlab # [ 6.642282] systemd[1]: Detected architecture arm64. vm-test-run-nixbot-gitlab> gitlab # [ 6.642424] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 6.643593] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 6.893568] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 7.017171] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 7.329530] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 7.574507] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.575220] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.578500] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 7.582455] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 7.584847] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.587323] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.588263] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.589067] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.589708] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 7.590309] systemd[1]: Expecting device /dev/ttyAMA0... vm-test-run-nixbot-gitlab> gitlab # [ 7.590926] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 7.591656] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.592314] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.592988] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.593637] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 7.594259] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.594868] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.595512] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.596118] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 7.598737] systemd[1]: Listening on Query the User Interactively for a Password. vm-test-run-nixbot-gitlab> gitlab # [ 7.601811] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.604244] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 7.606621] systemd[1]: Listening on Factory Reset Management. vm-test-run-nixbot-gitlab> gitlab # [ 7.608129] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.613886] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.615575] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.618774] systemd[1]: Listening on Console Output Muting Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.620678] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.621496] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 7.622164] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.626143] systemd[1]: Listening on Disk Repartitioning Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.627133] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.628137] systemd[1]: Listening on udev Varlink Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.656769] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.660837] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.666037] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.670844] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.680762] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 7.685838] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 7.686666] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 7.694932] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.695876] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 7.697466] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.704280] systemd[1]: Starting Load Kernel Module fuse... vm-test-run-nixbot-gitlab> gitlab # [ 7.705169] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.713305] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 7.718170] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 7.722945] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 7.756705] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 7.758521] systemd-journald[319]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 7.759518] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 7.760460] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 7.764742] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 7.778258] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.780358] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.781336] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.782065] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.783613] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 7.784810] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.785719] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 7.786563] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.784577] systemd[1]: Queued start job for default target Multi-User System.[ 7.787519] systemd[1]: modprobe@fuse.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 7.788459] systemd[1]: Finished Load Kernel Module fuse. vm-test-run-nixbot-gitlab> gitlab # [ 7.789278] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 7.798534] systemd[1]: Unnecessary job was removed for /dev/vdb. vm-test-run-nixbot-gitlab> gitlab # [ 7.802462] systemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.804054] systemd-modules-load[320]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 7.808880] systemd-modules-load[320]: Module 'atkbd' is built in vm-test-run-nixbot-gitlab> gitlab # [ 7.810088] systemd-modules-load[320]: Module 'loop' is built in vm-test-run-nixbot-gitlab> gitlab # [ 7.811351] systemd-modules-load[320]: Inserted module 'tls' vm-test-run-nixbot-gitlab> gitlab # [ 7.812506] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 7.813508] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.814527] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 7.815759] systemd-oomd[322]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 7.817506] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 7.825339] EXT4-fs (vda): re-mounted 76bf97d0-0d0a-442f-a3a7-ef255ef0c21a. vm-test-run-nixbot-gitlab> gitlab # [ 7.824499] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 7.826621] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 7.830997] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.832677] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.836307] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.846386] systemd[1]: Listening on Disk Image Download Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.850931] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 7.854962] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.857845] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 7.859375] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 7.869866] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 7.875776] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 7.880458] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 7.890826] systemd-journald[319]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 7.931355] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 7.933851] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 7.935019] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 7.936064] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.937130] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 7.938274] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 7.986744] systemd-udevd[356]: Using default interface naming scheme 'v261'. vm-test-run-nixbot-gitlab> gitlab # [ 8.035612] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 8.339665] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 8.486480] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 8.531203] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs vm-test-run-nixbot-gitlab> gitlab # [ 8.596197] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.605039] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.615310] (udev-worker)[398]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 8.619319] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.627898] (udev-worker)[397]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 8.632138] (udev-worker)[397]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 8.651771] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.657158] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 8.665594] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.670298] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.672372] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.673949] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 8.697225] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 8.700514] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 8.711291] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 8.719391] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 8.719070] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 8.720950] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 8.722391] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 8.724830] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 8.728777] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 8.731849] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os vm-test-run-nixbot-gitlab> gitlab # [ 8.764731] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.814695] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.829922] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.840137] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 8.841283] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 8.881755] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 8.903802] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 8.909782] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 8.955349] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 8.964989] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.967920] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.970145] systemd[1]: suid-sgid-wrappers.service: Consumed 191ms CPU time over 294ms wall clock time, 6M memory peak, 1.9M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 9.363691] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 9.434390] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.436814] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 9.439141] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 9.441581] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 9.442451] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 9.443481] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 9.444450] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 9.453869] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 9.499451] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 9.500849] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 9.502106] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 9.503924] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 9.509890] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 9.523623] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 9.567565] dbus-broker-launch[568]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 9.569376] dbus-broker-launch[568]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 9.571215] dbus-broker-launch[568]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 9.574235] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 9.590918] systemd-vconsole-setup[569]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 9.597238] dbus-broker-launch[568]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 9.598452] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 9.601982] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 9.604833] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.607529] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 9.611413] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 9.617102] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 9.653210] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 9.662858] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # [ 9.666464] logrotate[576]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 9.674472] logrotate[576]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 9.677197] logrotate[576]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 9.679033] logrotate[576]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # [ 9.680775] logrotate[576]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # [ 9.682315] logrotate[576]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 9.684362] logrotate[576]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 9.685566] logrotate[576]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.689180] logrotate[576]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab # [ 9.690444] logrotate[576]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.691433] logrotate[576]: Now: 2026-11-07 10:58 vm-test-run-nixbot-gitlab> gitlab # [ 9.692624] logrotate[576]: Last rotated at 2026-11-07 10:00 vm-test-run-nixbot-gitlab> gitlab # [ 9.693877] logrotate[576]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 9.695586] logrotate[576]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.699344] logrotate[576]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 9.700848] logrotate[576]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.701802] logrotate[576]: Now: 2026-11-07 10:58 vm-test-run-nixbot-gitlab> gitlab # [ 9.702812] logrotate[576]: Last rotated at 2026-11-07 10:00 vm-test-run-nixbot-gitlab> gitlab # [ 9.703996] logrotate[576]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 9.705843] logrotate[576]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.708180] logrotate[576]: switching euid from 0 to 165 and egid from 0 to 165 (pid 576) vm-test-run-nixbot-gitlab> gitlab # [ 9.709836] logrotate[576]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab # [ 9.711205] logrotate[576]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 9.712766] logrotate[576]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 9.713627] logrotate[576]: switching euid from 165 to 0 and egid from 165 to 0 (pid 576) vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab # [ 9.715144] logrotate[576]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 9.720386] logrotate[576]: switching euid from 0 to 60 and egid from 0 to 60 (pid 576) vm-test-run-nixbot-gitlab> gitlab # [ 9.721729] logrotate[576]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 9.722852] logrotate[576]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab # [ 9.724119] logrotate[576]: Creating new state vm-test-run-nixbot-gitlab> gitlab: (connecting took 10.23 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 9.724870] logrotate[576]: not running postrotate script, since no logs were rotated vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 10.23 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 9.726272] logrotate[576]: switching euid from 60 to 0 and egid from 60 to 0 (pid 576) vm-test-run-nixbot-gitlab> gitlab # [ 9.729062] nsncd[577]: Nov 07 10:58:44.450 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 9.731874] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 9.740539] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab # [ 9.741829] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 9.743328] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.744601] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 9.745456] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 9.747028] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.748262] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.758900] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 9.823849] systemd-logind[613]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 9.831200] systemd-logind[613]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 9.832774] systemd-logind[613]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 9.834120] systemd-logind[613]: Watching system buttons on /dev/input/event0 (gpio-keys) vm-test-run-nixbot-gitlab> gitlab # [ 9.835744] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 9.839451] systemd[1]: Stopped target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.840849] systemd[1]: Stopping Host and Network Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 9.841819] systemd[1]: Stopped target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.842765] systemd[1]: Stopping User and Group Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 9.845485] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 9.846445] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 9.852146] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.854294] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.855249] systemd[1]: nscd.service: Consumed 43ms CPU time over 197ms wall clock time, 7.5M memory peak, 5.8M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 9.869368] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 9.885354] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.887511] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.892692] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 9.893516] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 9.900128] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 9.901023] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 9.903337] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 9.909847] nsncd[674]: Nov 07 10:58:44.633 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 9.912365] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 9.913375] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.914410] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 9.968576] network-addresses-eth1-start[684]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 9.972619] network-addresses-eth1-start[684]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 9.980993] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 10.033123] dhcpcd[701]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 10.035254] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 10.036461] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 10.037572] dhcpcd[758]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 10.041305] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.045019] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 10.049872] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 10.060407] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 10.061205] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 10.067006] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 10.073004] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 10.085819] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 10.111119] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 10.129774] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 10.132461] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 10.134303] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 10.144736] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 10.145335] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 10.147293] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 10.147715] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # [ 10.195278] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 10.193988] dhcpcd[758]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 10.194853] dhcpcd[758]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 10.198564] dhcpcd[758]: DUID 00:01:00:01:32:81:c0:e4:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 10.200514] dhcpcd[758]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 10.201185] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 10.283406] nginx-pre-start[791]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 10.285571] nginx-pre-start[791]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 10.291019] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 10.309169] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 10.312732] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 10.313985] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started vm-test-run-nixbot-gitlab> gitlab # [ 10.315677] redis-server[788]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 10.316630] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 10.317514] redis[788]: monotonic clock: ARM CNTVCT @ 1000 ticks/us vm-test-run-nixbot-gitlab> gitlab # [ 10.318590] redis[788]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 10.320530] redis[788]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 10.321487] redis[788]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 10.322733] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 10.338654] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 10.340378] postgresql-pre-start[796]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 10.346748] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 10.348583] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 10.350219] postgresql-pre-start[796]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 10.351710] postgresql-pre-start[796]: Data page checksums are enabled. vm-test-run-nixbot-gitlab> gitlab # [ 10.352921] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.354415] postgresql-pre-start[796]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.355478] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 10.408572] postgresql-pre-start[796]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 10.465940] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 11.031461] postgresql-pre-start[796]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 11.034988] postgresql-pre-start[796]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 11.081197] dhcpcd[758]: eth0: soliciting a DHCP lease vm-test-run-nixbot-gitlab> gitlab # [ 11.095899] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 11.120522] dhcpcd[758]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 11.128598] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.342960] postgresql-pre-start[796]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 11.533317] gitlab-workhorse[830]: time="2026-11-07T10:58:46Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 vm-test-run-nixbot-gitlab> gitlab # [ 11.535138] gitlab-workhorse[830]: time="2026-11-07T10:58:46Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 11.538955] gitlab-workhorse[830]: time="2026-11-07T10:58:46Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 12.153450] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 12.425084] postgresql-pre-start[796]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 12.426729] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 12.429082] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 12.432602] postgresql-pre-start[796]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 12.434461] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 12.573700] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 12.578080] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 12.580127] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 12.582226] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 12.597683] postgres[863]: [863] LOG: database system was shut down at 2026-11-07 10:58:46 GMT vm-test-run-nixbot-gitlab> gitlab # [ 12.606550] postgres[854]: [854] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 12.610857] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 12.623994] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 12.888941] dhcpcd[758]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 12.890550] dhcpcd[758]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 12.891965] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 12.893503] dhcpcd[758]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 12.894688] dhcpcd[758]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 12.923742] postgresql-setup-start[874]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 12.963167] postgresql-setup-start[883]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 12.984914] postgresql-setup-start[889]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 13.025918] postgresql-setup-start[900]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 13.029670] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 13.031717] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 13.035048] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 13.233521] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[879]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 13.235623] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[879]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 13.238433] gitlab-postgresql-start[908]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 13.300337] gitlab-postgresql-start[918]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 13.317275] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 13.321984] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 13.326546] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 13.431695] gitlab-postgresql-start[925]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 13.437337] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 14.260693] gitaly[928]: time="2026-11-07T10:58:48.983Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.260930] gitaly[928]: time="2026-11-07T10:58:48.984Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.260991] gitaly[928]: time="2026-11-07T10:58:48.984Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.261040] gitaly[928]: time="2026-11-07T10:58:48.984Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.261089] gitaly[928]: time="2026-11-07T10:58:48.984Z" level=info msg="Starting Gitaly" pid=928 version= vm-test-run-nixbot-gitlab> gitlab # [ 14.541445] gitaly[928]: time="2026-11-07T10:58:49.265Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=280 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.544503] gitaly[928]: time="2026-11-07T10:58:49.265Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.546975] gitaly[928]: time="2026-11-07T10:58:49.265Z" level=info msg="finished initializing command factory" duration_ms=0 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.549682] gitaly[928]: time="2026-11-07T10:58:49.265Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-928/git-exec-526163287.d/git pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.552817] gitaly[928]: time="2026-11-07T10:58:49.272Z" level=info msg="finished detecting git version" duration_ms=6 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.555188] gitaly[928]: time="2026-11-07T10:58:49.272Z" level=info msg="using Git version" pid=928 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 14.557606] gitaly[928]: time="2026-11-07T10:58:49.272Z" level=info msg="clearing disk cache object folder" pid=928 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 14.560410] gitaly[928]: time="2026-11-07T10:58:49.273Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2048898237 pid=928 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 14.564739] gitaly[928]: time="2026-11-07T10:58:49.273Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2048898237 pid=928 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 14.570379] gitaly[928]: time="2026-11-07T10:58:49.273Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.574509] gitaly[928]: time="2026-11-07T10:58:49.273Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.580816] gitaly[928]: time="2026-11-07T10:58:49.273Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2048898237 clear_duration_ms=0 pid=928 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 14.584578] gitaly[928]: time="2026-11-07T10:58:49.295Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.587621] gitaly[928]: time="2026-11-07T10:58:49.295Z" level=info msg="counting repositories" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.589466] gitaly[928]: time="2026-11-07T10:58:49.295Z" level=info msg="starting to count repositories" pid=928 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 14.592174] gitaly[928]: time="2026-11-07T10:58:49.295Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=928 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 14.594963] gitaly[928]: time="2026-11-07T10:58:49.296Z" level=info msg="listening at address" address=/run/gitaly/gitaly-928/sock.d/intern pid=928 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 14.597480] gitaly[928]: time="2026-11-07T10:58:49.296Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=928 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 14.600293] gitaly[928]: time="2026-11-07T10:58:49.296Z" level=info msg="completed counting all repositories" counting_duration=0 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.602455] gitaly[928]: time="2026-11-07T10:58:49.296Z" level=info msg="finished tempdir cleaner walk" pid=928 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 14.604835] gitaly[928]: time="2026-11-07T10:58:49.297Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.607133] gitaly[928]: time="2026-11-07T10:58:49.297Z" level=info msg="Started Gitaly" duration_ms=313 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.609001] gitaly[928]: time="2026-11-07T10:58:49.298Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 14.611134] gitaly[928]: time="2026-11-07T10:58:49.298Z" level=info msg="maintenance: daily scheduled" pid=928 scheduled="2026-11-07 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 15.978464] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[931]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 15.979715] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[931]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 16.033086] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[931]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 16.133024] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 16.134536] dhcpcd[758]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 16.135843] dhcpcd[758]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 16.281762] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 16.282736] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 18.808176] gitaly[928]: time="2026-11-07T10:58:53.531Z" level=info msg="License database preloaded" duration_ms=4546 pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 85.688825] systemd[1]: Starting Logrotate Service... vm-test-run-nixbot-gitlab> gitlab # [ 85.833180] systemd[1]: logrotate.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 85.835234] systemd[1]: Finished Logrotate Service. vm-test-run-nixbot-gitlab> gitlab # [ 88.553765] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[931]: W, [2026-11-07T11:00:03.276942 #931] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 91.642946] gitlab-workhorse[830]: time="2026-11-07T11:00:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 92.744534] postgres[1712]: [1712] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 92.744831] postgres[1712]: [1712] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 92.744910] postgres[1712]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 92.744963] postgres[1712]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 92.745022] postgres[1712]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 92.745071] postgres[1712]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 92.745147] postgres[1712]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 92.745209] postgres[1712]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 92.745261] postgres[1712]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 92.745321] postgres[1712]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 92.745380] postgres[1712]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 92.745439] postgres[1712]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 92.745497] postgres[1712]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 92.745555] postgres[1712]: vm-test-run-nixbot-gitlab> gitlab # [ 100.521957] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1782]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 100.522216] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1782]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 101.644663] gitlab-workhorse[830]: time="2026-11-07T11:00:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 111.654311] gitlab-workhorse[830]: time="2026-11-07T11:00:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 121.664284] gitlab-workhorse[830]: time="2026-11-07T11:00:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 131.314843] postgres[1939]: [1939] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 131.315144] postgres[1939]: [1939] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 131.666749] gitlab-workhorse[830]: time="2026-11-07T11:00:46Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 141.667485] gitlab-workhorse[830]: time="2026-11-07T11:00:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 148.936888] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 148.941998] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 148.945711] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 149.684592] bundle[2205]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 149.684927] bundle[2205]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 149.688007] bundle[2205]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 151.677715] gitlab-workhorse[830]: time="2026-11-07T11:01:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 157.154038] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.827Z","pid":2205,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154435] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 157.154512] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154574] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154633] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154703] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154763] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154820] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Master PID: 2205"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154877] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154934] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 157.154982] bundle[2205]: {"timestamp":"2026-11-07T11:01:11.877Z","pid":2205,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 157.231315] sidekiq-cluster[2204]: {"severity":"INFO","time":"2026-11-07T11:01:11.904Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 161.685552] gitlab-workhorse[830]: time="2026-11-07T11:01:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 171.686408] gitlab-workhorse[830]: time="2026-11-07T11:01:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 181.697282] gitlab-workhorse[830]: time="2026-11-07T11:01:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 191.705789] gitlab-workhorse[830]: time="2026-11-07T11:01:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 201.706487] gitlab-workhorse[830]: time="2026-11-07T11:01:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 211.713601] gitlab-workhorse[830]: time="2026-11-07T11:02:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 214.377245] bundle[2205]: W, [2026-11-07T11:02:09.100362 #2205] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 221.721700] gitlab-workhorse[830]: time="2026-11-07T11:02:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 223.605950] sidekiq-cluster[2281]: 2026-11-07T11:02:18.329Z pid=2281 tid=xmp INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2281, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>427417600} vm-test-run-nixbot-gitlab> gitlab # [ 224.043744] sidekiq-cluster[2281]: W, [2026-11-07T11:02:18.767646 #2281] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 224.245863] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:18.969Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 229.003679] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.726Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 229.004130] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.727Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 229.059932] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.783Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.067371] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.791Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.076622] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.800Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.085381] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.809Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.094780] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.818Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.102716] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.826Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.109593] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.833Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.118187] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.842Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.126477] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.850Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.133129] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.857Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.141274] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.865Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.149543] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.873Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.156953] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.880Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.165349] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.889Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.171990] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.895Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.175985] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.899Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.181363] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.905Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.189243] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.913Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.194663] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.918Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.200684] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.924Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.205614] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.929Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.209750] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.932Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.213411] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.937Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.217971] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.941Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.222301] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.946Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.227527] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.951Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.232128] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.956Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.237722] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.961Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.242996] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.966Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.246721] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.970Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.250246] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.973Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.254063] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.977Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.258105] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.982Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.262564] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.986Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.266402] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.990Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.269678] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.993Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.273747] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:23.997Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.277692] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.001Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.280967] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.004Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.284331] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.008Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.288205] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.012Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.291843] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.015Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.295171] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.019Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.299101] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.023Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.303869] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.027Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.307133] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.031Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.310156] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.033Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.313098] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.036Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.315815] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.039Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.319053] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.043Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.323092] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.047Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.328706] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.052Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.343955] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.067Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.347686] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.071Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.351094] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.075Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.355107] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.079Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.358334] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.082Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.361750] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.085Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.364800] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.088Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.367857] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.090Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.370859] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.094Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.473671] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.197Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.478013] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.201Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.481906] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.205Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.486790] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.210Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.490489] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.214Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.493925] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.217Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.497550] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.220Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.501319] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.224Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.505182] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.226Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.508803] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.229Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.512425] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.233Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.516073] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.235Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.519973] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.240Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.523948] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.242Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.527916] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.246Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.532678] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.248Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.536284] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.252Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.539551] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.254Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.542822] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.257Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.546388] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.261Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.550023] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.263Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.553448] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.268Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.556703] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.270Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.559963] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.274Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.563172] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.277Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.568209] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.281Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.571285] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.284Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.574570] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.287Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.577851] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.290Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.581038] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.297Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.583965] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.300Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.587129] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.305Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.590257] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.313Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.598560] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.322Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.605789] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.329Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.615799] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.339Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.622333] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.346Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.635679] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.359Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.639587] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.362Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.643262] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.366Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.646868] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.370Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.650735] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.374Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.654225] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.378Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.657898] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.381Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.661433] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.384Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.664750] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.386Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.667936] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.390Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 229.671159] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:02:24.392Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 231.729746] gitlab-workhorse[830]: time="2026-11-07T11:02:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 241.737829] gitlab-workhorse[830]: time="2026-11-07T11:02:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 251.739232] gitlab-workhorse[830]: time="2026-11-07T11:02:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 261.741449] gitlab-workhorse[830]: time="2026-11-07T11:02:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 271.742293] gitlab-workhorse[830]: time="2026-11-07T11:03:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 281.429908] postgres[3363]: [3363] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 281.430198] postgres[3363]: [3363] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 281.752460] gitlab-workhorse[830]: time="2026-11-07T11:03:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 282.774970] postgres[3374]: [3374] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 282.775223] postgres[3374]: [3374] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 287.200906] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:21.922Z","message":"Booted Rails 7.2.3.1 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 287.201177] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:21.923Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 287.201247] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:21.923Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 287.201327] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:21.924Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 287.216494] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:21.933Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 289.339840] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.062Z","pid":2205,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 289.340292] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.063Z","pid":2205,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 289.340434] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.063Z","pid":2205,"message":"! #\u003cThread:0x0000e67691ac49e8 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 289.340532] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.063Z","pid":2205,"message":"! #\u003cThread:0x0000e67691d3e3b8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 289.340661] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.063Z","pid":2205,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 289.341031] bundle[2205]: {"timestamp":"2026-11-07T11:03:24.064Z","pid":2205,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 290.323683] bundle[2205]: {"timestamp":"2026-11-07T11:03:25.046Z","pid":2205,"message":"- Worker 0 (PID: 3475) booted in 0.89s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 290.359156] bundle[2205]: {"timestamp":"2026-11-07T11:03:25.082Z","pid":2205,"message":"- Worker 1 (PID: 3479) booted in 0.89s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 290.369700] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 290.370602] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 290.375285] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 290.409808] setup-gitlab-start[3512]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 290.933968] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:25.657Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 290.942293] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:25.665Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2b526071845d94272a9ccb22","created_at":"2026-11-07T11:01:02.932Z","trace_propagation_headers":{"sentry-trace":"b40051a25e654a2ea098ade80a7883f8-2f7c0c9da245441a","baggage":"sentry-trace_id=b40051a25e654a2ea098ade80a7883f8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e5968113b711c6f1f58d937b7207f1bc","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-07T11:01:02.947Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2b526071845d94272a9ccb22: start","job_status":"start","queue_duration_s":142.718328,"scheduling_latency_s":142.718412} vm-test-run-nixbot-gitlab> gitlab # [ 291.319800] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:26.042Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2b526071845d94272a9ccb22","created_at":"2026-11-07T11:01:02.932Z","trace_propagation_headers":{"sentry-trace":"b40051a25e654a2ea098ade80a7883f8-2f7c0c9da245441a","baggage":"sentry-trace_id=b40051a25e654a2ea098ade80a7883f8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e5968113b711c6f1f58d937b7207f1bc","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-07T11:01:02.947Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2b526071845d94272a9ccb22: done: 0.377643 sec","job_status":"done","queue_duration_s":142.718328,"scheduling_latency_s":142.718412,"redis_calls":24,"redis_duration_s":0.010689,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005539,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002854,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002296,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.18571624,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.377643,"completed_at":"2026-11-07T11:03:26.042Z","load_balancing_strategy":"primary","db_duration_s":0.089521,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 296.249585] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:03:30 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 5792 vm-test-run-nixbot-gitlab> gitlab # [ 296.254175] setup-gitlab-start[3512]: + TOKEN=nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 296.256151] setup-gitlab-start[3512]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' vm-test-run-nixbot-gitlab> gitlab # [ 296.290072] runuser[3558]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 297.093709] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 297.094049] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 297.098498] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 298.324151] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 298.324503] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 298.328136] setup-gitlab-start[3559]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 305.572630] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:40.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c8fefdf1ce5e385781274494","created_at":"2026-11-07T11:03:40.252Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"911ab46f147f20ec8837555fb1e4b6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:03:40.285Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c8fefdf1ce5e385781274494: start","job_status":"start","queue_duration_s":0.010209,"scheduling_latency_s":0.01025} vm-test-run-nixbot-gitlab> gitlab # [ 305.721987] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:40.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c8fefdf1ce5e385781274494","created_at":"2026-11-07T11:03:40.252Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"911ab46f147f20ec8837555fb1e4b6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:03:40.285Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c8fefdf1ce5e385781274494: done: 0.149906 sec","job_status":"done","queue_duration_s":0.010209,"scheduling_latency_s":0.01025,"redis_calls":10,"redis_duration_s":0.01564,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002118,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00299792,"duration_s":0.149906,"completed_at":"2026-11-07T11:03:40.445Z","load_balancing_strategy":"primary","db_duration_s":0.047074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 305.785811] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:40.458Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9c9db4d636ecdc59259488d3","created_at":"2026-11-07T11:03:40.297Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3540d4ca7d56f438d8b4f55492eb93d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.402Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9c9db4d636ecdc59259488d3: start","job_status":"start","queue_duration_s":0.056012,"scheduling_latency_s":0.056059} vm-test-run-nixbot-gitlab> gitlab # [ 305.966260] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:40.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9c9db4d636ecdc59259488d3","created_at":"2026-11-07T11:03:40.297Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3540d4ca7d56f438d8b4f55492eb93d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.402Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9c9db4d636ecdc59259488d3: done: 0.230633 sec","job_status":"done","queue_duration_s":0.056012,"scheduling_latency_s":0.056059,"redis_calls":16,"redis_duration_s":0.07592,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.013938,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.059127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002855,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00378162,"duration_s":0.230633,"completed_at":"2026-11-07T11:03:40.689Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.031919] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:40.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fe6d56b08cd890761a2a2f8d","created_at":"2026-11-07T11:03:40.426Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87e57a66d81a97716c474b7646dac568","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.450Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fe6d56b08cd890761a2a2f8d: start","job_status":"start","queue_duration_s":0.244909,"scheduling_latency_s":0.244959} vm-test-run-nixbot-gitlab> gitlab # [ 306.325849] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.047Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fe6d56b08cd890761a2a2f8d","created_at":"2026-11-07T11:03:40.426Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87e57a66d81a97716c474b7646dac568","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.450Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fe6d56b08cd890761a2a2f8d: done: 0.352565 sec","job_status":"done","queue_duration_s":0.244909,"scheduling_latency_s":0.244959,"redis_calls":12,"redis_duration_s":0.023828,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001102,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003046,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.119144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00170296,"exclusive_lock_hold_duration_s":0.21503058,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.352565,"completed_at":"2026-11-07T11:03:41.047Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.395175] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.059Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"885b4cc5e5f87e7594586703","created_at":"2026-11-07T11:03:40.457Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cff4440a0a5ca91d5823aca6b7c28313","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.489Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-885b4cc5e5f87e7594586703: start","job_status":"start","queue_duration_s":0.570005,"scheduling_latency_s":0.570061} vm-test-run-nixbot-gitlab> gitlab # [ 306.412329] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.121Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 306.433935] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"885b4cc5e5f87e7594586703","created_at":"2026-11-07T11:03:40.457Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cff4440a0a5ca91d5823aca6b7c28313","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.489Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-885b4cc5e5f87e7594586703: done: 0.097456 sec","job_status":"done","queue_duration_s":0.570005,"scheduling_latency_s":0.570061,"redis_calls":10,"redis_duration_s":0.03252,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005014,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00938,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00180892,"duration_s":0.097456,"completed_at":"2026-11-07T11:03:41.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.494419] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"37dba6af93b7f224cd2abea7","created_at":"2026-11-07T11:03:40.532Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc7b520a60ce250903690a5a413dca5a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.564Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-37dba6af93b7f224cd2abea7: start","job_status":"start","queue_duration_s":0.597345,"scheduling_latency_s":0.597442} vm-test-run-nixbot-gitlab> gitlab # [ 306.618495] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"37dba6af93b7f224cd2abea7","created_at":"2026-11-07T11:03:40.532Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc7b520a60ce250903690a5a413dca5a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.564Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-37dba6af93b7f224cd2abea7: done: 0.179355 sec","job_status":"done","queue_duration_s":0.597345,"scheduling_latency_s":0.597442,"redis_calls":10,"redis_duration_s":0.053182,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.043018,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0010422,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.179355,"completed_at":"2026-11-07T11:03:41.340Z","load_balancing_strategy":"primary","db_duration_s":0.04351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.681615] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dedc683885cf555579acb88b","created_at":"2026-11-07T11:03:40.574Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"607b226d42c8af942f42b5de3cbd549e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.591Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dedc683885cf555579acb88b: start","job_status":"start","queue_duration_s":0.768213,"scheduling_latency_s":0.768368} vm-test-run-nixbot-gitlab> gitlab # [ 306.712068] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.435Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 306.730501] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dedc683885cf555579acb88b","created_at":"2026-11-07T11:03:40.574Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"607b226d42c8af942f42b5de3cbd549e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.591Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dedc683885cf555579acb88b: done: 0.093573 sec","job_status":"done","queue_duration_s":0.768213,"scheduling_latency_s":0.768368,"redis_calls":10,"redis_duration_s":0.012193,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000842,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008961,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062846,"duration_s":0.093573,"completed_at":"2026-11-07T11:03:41.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.791249] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4771cd5dcfdccbe157ddd540","created_at":"2026-11-07T11:03:40.616Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"711a4762a06c995cadece06e98b88930","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:03:40.641Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4771cd5dcfdccbe157ddd540: start","job_status":"start","queue_duration_s":0.81599,"scheduling_latency_s":0.81604} vm-test-run-nixbot-gitlab> gitlab # [ 306.806039] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.491Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 306.809959] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4771cd5dcfdccbe157ddd540","created_at":"2026-11-07T11:03:40.616Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"711a4762a06c995cadece06e98b88930","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:03:40.641Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4771cd5dcfdccbe157ddd540: done: 0.050297 sec","job_status":"done","queue_duration_s":0.81599,"scheduling_latency_s":0.81604,"redis_calls":10,"redis_duration_s":0.00362,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000749,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044068,"duration_s":0.050297,"completed_at":"2026-11-07T11:03:41.507Z","load_balancing_strategy":"primary","db_duration_s":0.001096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 306.852346] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:41.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"436ce7d6a1bd5cb7ef3755ae","created_at":"2026-11-07T11:03:40.667Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c461b3fad01c256552ba58ca8d2f7a7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.699Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-436ce7d6a1bd5cb7ef3755ae: start","job_status":"start","queue_duration_s":0.810908,"scheduling_latency_s":0.810951} vm-test-run-nixbot-gitlab> gitlab # [ 307.294509] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"436ce7d6a1bd5cb7ef3755ae","created_at":"2026-11-07T11:03:40.667Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c461b3fad01c256552ba58ca8d2f7a7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.699Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-436ce7d6a1bd5cb7ef3755ae: done: 0.508054 sec","job_status":"done","queue_duration_s":0.810908,"scheduling_latency_s":0.810951,"redis_calls":565,"redis_duration_s":0.107708,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000665,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.105534,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.451656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036292,"duration_s":0.508054,"completed_at":"2026-11-07T11:03:42.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 307.363165] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"21b5819511796a289de67c6e","created_at":"2026-11-07T11:03:40.764Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"442f1606bc3f891d491b7a67fdfdc24e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:03:40.790Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-21b5819511796a289de67c6e: start","job_status":"start","queue_duration_s":1.229463,"scheduling_latency_s":1.229517} vm-test-run-nixbot-gitlab> gitlab # [ 307.381050] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.062Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 307.384967] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"21b5819511796a289de67c6e","created_at":"2026-11-07T11:03:40.764Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"442f1606bc3f891d491b7a67fdfdc24e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:03:40.790Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-21b5819511796a289de67c6e: done: 0.055034 sec","job_status":"done","queue_duration_s":1.229463,"scheduling_latency_s":1.229517,"redis_calls":10,"redis_duration_s":0.00292,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001567,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052484,"duration_s":0.055034,"completed_at":"2026-11-07T11:03:42.074Z","load_balancing_strategy":"primary","db_duration_s":0.021468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 307.428602] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"021a7c875ba9847b7237637c","created_at":"2026-11-07T11:03:40.882Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acb4f2c34bb26d87db1fb2609740372c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.897Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-021a7c875ba9847b7237637c: start","job_status":"start","queue_duration_s":1.180206,"scheduling_latency_s":1.180241} vm-test-run-nixbot-gitlab> gitlab # [ 307.443679] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"021a7c875ba9847b7237637c","created_at":"2026-11-07T11:03:40.882Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acb4f2c34bb26d87db1fb2609740372c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.897Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-021a7c875ba9847b7237637c: done: 0.078548 sec","job_status":"done","queue_duration_s":1.180206,"scheduling_latency_s":1.180241,"redis_calls":13,"redis_duration_s":0.004673,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002375,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001686,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02369044,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.078548,"completed_at":"2026-11-07T11:03:42.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022686,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 307.490026] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.160Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4d417f36bef13fcde7164586","created_at":"2026-11-07T11:03:40.905Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7b04ce17701a6f88004c66f104cedd2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.916Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4d417f36bef13fcde7164586: start","job_status":"start","queue_duration_s":1.243583,"scheduling_latency_s":1.243659} vm-test-run-nixbot-gitlab> gitlab # [ 307.504330] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4d417f36bef13fcde7164586","created_at":"2026-11-07T11:03:40.905Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7b04ce17701a6f88004c66f104cedd2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.916Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4d417f36bef13fcde7164586: done: 0.057844 sec","job_status":"done","queue_duration_s":1.243583,"scheduling_latency_s":1.243659,"redis_calls":12,"redis_duration_s":0.004161,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0009735,"exclusive_lock_hold_duration_s":0.01823684,"duration_s":0.057844,"completed_at":"2026-11-07T11:03:42.217Z","load_balancing_strategy":"primary","db_duration_s":0.00939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 307.548692] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d3e5fc444f3f6588c8eade76","created_at":"2026-11-07T11:03:40.927Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11355a353b0efb7cea8b8b13b603662f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.938Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d3e5fc444f3f6588c8eade76: start","job_status":"start","queue_duration_s":1.281381,"scheduling_latency_s":1.281415} vm-test-run-nixbot-gitlab> gitlab # [ 307.850562] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.573Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d3e5fc444f3f6588c8eade76","created_at":"2026-11-07T11:03:40.927Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11355a353b0efb7cea8b8b13b603662f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.938Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d3e5fc444f3f6588c8eade76: done: 0.35311 sec","job_status":"done","queue_duration_s":1.281381,"scheduling_latency_s":1.281415,"redis_calls":14,"redis_duration_s":0.003525,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000901,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.052,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.283176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007802,"exclusive_lock_hold_duration_s":0.3193052,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.35311,"completed_at":"2026-11-07T11:03:42.573Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 307.919443] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0b9862ddc9e8272f35aa0de","created_at":"2026-11-07T11:03:40.945Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.963Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0b9862ddc9e8272f35aa0de: start","job_status":"start","queue_duration_s":1.612131,"scheduling_latency_s":1.61216} vm-test-run-nixbot-gitlab> gitlab # [ 308.036181] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0b9862ddc9e8272f35aa0de","created_at":"2026-11-07T11:03:40.945Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:40.963Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0b9862ddc9e8272f35aa0de: done: 0.184306 sec","job_status":"done","queue_duration_s":1.612131,"scheduling_latency_s":1.61216,"redis_calls":71,"redis_duration_s":0.021407,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002515,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001074,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.017106,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.164005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039644,"duration_s":0.184306,"completed_at":"2026-11-07T11:03:42.759Z","load_balancing_strategy":"primary","db_duration_s":0.002725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.103363] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"73a591634839356f3b2f68da","created_at":"2026-11-07T11:03:40.979Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f160708d18b59c21ae5a20b87411fca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.000Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-73a591634839356f3b2f68da: start","job_status":"start","queue_duration_s":1.760746,"scheduling_latency_s":1.760783} vm-test-run-nixbot-gitlab> gitlab # [ 308.121556] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.785Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 308.125408] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"73a591634839356f3b2f68da","created_at":"2026-11-07T11:03:40.979Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f160708d18b59c21ae5a20b87411fca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.000Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-73a591634839356f3b2f68da: done: 0.035935 sec","job_status":"done","queue_duration_s":1.760746,"scheduling_latency_s":1.760783,"redis_calls":10,"redis_duration_s":0.003289,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00061,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038834,"duration_s":0.035935,"completed_at":"2026-11-07T11:03:42.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.170908] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"089a3976f6f0ea057ce3f53f","created_at":"2026-11-07T11:03:41.037Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80d98e6833438469ab15a62ef78187c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.056Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-089a3976f6f0ea057ce3f53f: start","job_status":"start","queue_duration_s":1.74381,"scheduling_latency_s":1.743844} vm-test-run-nixbot-gitlab> gitlab # [ 308.186143] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.855Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"089a3976f6f0ea057ce3f53f","created_at":"2026-11-07T11:03:41.037Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80d98e6833438469ab15a62ef78187c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.056Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-089a3976f6f0ea057ce3f53f: done: 0.056049 sec","job_status":"done","queue_duration_s":1.74381,"scheduling_latency_s":1.743844,"redis_calls":12,"redis_duration_s":0.010457,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001304,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00162,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050602,"duration_s":0.056049,"completed_at":"2026-11-07T11:03:42.855Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.230310] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"fb246c458a0beb374b0c7209","created_at":"2026-11-07T11:03:41.087Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af3e95dafbc405afc451a8316d3b4e27","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.117Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-fb246c458a0beb374b0c7209: start","job_status":"start","queue_duration_s":1.740736,"scheduling_latency_s":1.740779} vm-test-run-nixbot-gitlab> gitlab # [ 308.243936] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"fb246c458a0beb374b0c7209","created_at":"2026-11-07T11:03:41.087Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af3e95dafbc405afc451a8316d3b4e27","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.117Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-fb246c458a0beb374b0c7209: done: 0.061399 sec","job_status":"done","queue_duration_s":1.740736,"scheduling_latency_s":1.740779,"redis_calls":12,"redis_duration_s":0.004578,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001148,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00111246,"exclusive_lock_hold_duration_s":0.02172376,"duration_s":0.061399,"completed_at":"2026-11-07T11:03:42.919Z","load_balancing_strategy":"primary","db_duration_s":0.00825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.286830] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"726188a41fefc7cce8f3f6ca","created_at":"2026-11-07T11:03:41.150Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14cb2015d1731c5c003d8543bf7764e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.174Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-726188a41fefc7cce8f3f6ca: start","job_status":"start","queue_duration_s":1.747601,"scheduling_latency_s":1.747641} vm-test-run-nixbot-gitlab> gitlab # [ 308.302092] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.944Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 308.306113] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"726188a41fefc7cce8f3f6ca","created_at":"2026-11-07T11:03:41.150Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14cb2015d1731c5c003d8543bf7764e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.174Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-726188a41fefc7cce8f3f6ca: done: 0.036328 sec","job_status":"done","queue_duration_s":1.747601,"scheduling_latency_s":1.747641,"redis_calls":10,"redis_duration_s":0.002903,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043514,"duration_s":0.036328,"completed_at":"2026-11-07T11:03:42.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.351396] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:42.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"91f1f307dc558f44b1f04a4a","created_at":"2026-11-07T11:03:41.207Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2fc2bdf3d734ee956edf5847aa72b3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.243Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-91f1f307dc558f44b1f04a4a: start","job_status":"start","queue_duration_s":1.719586,"scheduling_latency_s":1.719632} vm-test-run-nixbot-gitlab> gitlab # [ 308.366188] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"91f1f307dc558f44b1f04a4a","created_at":"2026-11-07T11:03:41.207Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d2fc2bdf3d734ee956edf5847aa72b3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.243Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-91f1f307dc558f44b1f04a4a: done: 0.045645 sec","job_status":"done","queue_duration_s":1.719586,"scheduling_latency_s":1.719632,"redis_calls":10,"redis_duration_s":0.003545,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000583,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071028,"duration_s":0.045645,"completed_at":"2026-11-07T11:03:43.008Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.409860] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ca5ba0ad12649694b7741030","created_at":"2026-11-07T11:03:41.251Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0653f0244e2f77523fd7732f6d32739e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.273Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ca5ba0ad12649694b7741030: start","job_status":"start","queue_duration_s":1.73747,"scheduling_latency_s":1.737507} vm-test-run-nixbot-gitlab> gitlab # [ 308.424894] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ca5ba0ad12649694b7741030","created_at":"2026-11-07T11:03:41.251Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0653f0244e2f77523fd7732f6d32739e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.273Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ca5ba0ad12649694b7741030: done: 0.039646 sec","job_status":"done","queue_duration_s":1.73747,"scheduling_latency_s":1.737507,"redis_calls":12,"redis_duration_s":0.003362,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00087536,"exclusive_lock_hold_duration_s":0.0059691,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.039646,"completed_at":"2026-11-07T11:03:43.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.469607] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b0fb35744933f02ba70d8a1d","created_at":"2026-11-07T11:03:41.309Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d459cce9d248b8f860c83dad2f8d5ece","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.334Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b0fb35744933f02ba70d8a1d: start","job_status":"start","queue_duration_s":1.71805,"scheduling_latency_s":1.718084} vm-test-run-nixbot-gitlab> gitlab # [ 308.484694] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b0fb35744933f02ba70d8a1d","created_at":"2026-11-07T11:03:41.309Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d459cce9d248b8f860c83dad2f8d5ece","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.334Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b0fb35744933f02ba70d8a1d: done: 0.034298 sec","job_status":"done","queue_duration_s":1.71805,"scheduling_latency_s":1.718084,"redis_calls":10,"redis_duration_s":0.002628,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003811,"duration_s":0.034298,"completed_at":"2026-11-07T11:03:43.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.528596] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"663e89cf7a9f9fe1b52fb944","created_at":"2026-11-07T11:03:41.348Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fa0c85f9204ac4a0634c8970bf55686","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.369Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-663e89cf7a9f9fe1b52fb944: start","job_status":"start","queue_duration_s":1.719404,"scheduling_latency_s":1.719436} vm-test-run-nixbot-gitlab> gitlab # [ 308.571900] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"663e89cf7a9f9fe1b52fb944","created_at":"2026-11-07T11:03:41.348Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fa0c85f9204ac4a0634c8970bf55686","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.369Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-663e89cf7a9f9fe1b52fb944: done: 0.206425 sec","job_status":"done","queue_duration_s":1.719404,"scheduling_latency_s":1.719436,"redis_calls":12,"redis_duration_s":0.0033179999999999998,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001559,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.179468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00071866,"exclusive_lock_hold_duration_s":0.02998082,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.206425,"completed_at":"2026-11-07T11:03:43.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.620670] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60c69fce2a1d7768dd9a7d85","created_at":"2026-11-07T11:03:41.386Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.398Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60c69fce2a1d7768dd9a7d85: start","job_status":"start","queue_duration_s":1.899185,"scheduling_latency_s":1.899216} vm-test-run-nixbot-gitlab> gitlab # [ 308.634699] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60c69fce2a1d7768dd9a7d85","created_at":"2026-11-07T11:03:41.386Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:03:41.398Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60c69fce2a1d7768dd9a7d85: done: 0.053681 sec","job_status":"done","queue_duration_s":1.899185,"scheduling_latency_s":1.899216,"redis_calls":19,"redis_duration_s":0.004667,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001192,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002239,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032886,"duration_s":0.053681,"completed_at":"2026-11-07T11:03:43.350Z","load_balancing_strategy":"primary","db_duration_s":0.003876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.679505] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"654b15bbeb68ac1e8108caab","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-654b15bbeb68ac1e8108caab: start","job_status":"start","queue_duration_s":0.609536,"scheduling_latency_s":0.609568} vm-test-run-nixbot-gitlab> gitlab # [ 308.692891] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"654b15bbeb68ac1e8108caab","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-654b15bbeb68ac1e8108caab: done: 0.039272 sec","job_status":"done","queue_duration_s":0.609536,"scheduling_latency_s":0.609568,"redis_calls":16,"redis_duration_s":0.009488,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007531,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042542,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039272,"completed_at":"2026-11-07T11:03:43.392Z","load_balancing_strategy":"primary","db_duration_s":0.000721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.735881] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cad3cd37d7dea6801dbedd4a","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cad3cd37d7dea6801dbedd4a: start","job_status":"start","queue_duration_s":0.65181,"scheduling_latency_s":0.651843} vm-test-run-nixbot-gitlab> gitlab # [ 308.749019] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cad3cd37d7dea6801dbedd4a","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cad3cd37d7dea6801dbedd4a: done: 0.031069 sec","job_status":"done","queue_duration_s":0.65181,"scheduling_latency_s":0.651843,"redis_calls":13,"redis_duration_s":0.006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00177,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031069,"completed_at":"2026-11-07T11:03:43.426Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.790611] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6be1a2f19e34e6efc140dedd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6be1a2f19e34e6efc140dedd: start","job_status":"start","queue_duration_s":0.686796,"scheduling_latency_s":0.686839} vm-test-run-nixbot-gitlab> gitlab # [ 308.803679] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6be1a2f19e34e6efc140dedd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6be1a2f19e34e6efc140dedd: done: 0.031623 sec","job_status":"done","queue_duration_s":0.686796,"scheduling_latency_s":0.686839,"redis_calls":13,"redis_duration_s":0.0032319999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031623,"completed_at":"2026-11-07T11:03:43.461Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.844669] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9907258a1d433abca2f84e4f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9907258a1d433abca2f84e4f: start","job_status":"start","queue_duration_s":0.720604,"scheduling_latency_s":0.720643} vm-test-run-nixbot-gitlab> gitlab # [ 308.857922] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9907258a1d433abca2f84e4f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9907258a1d433abca2f84e4f: done: 0.03016 sec","job_status":"done","queue_duration_s":0.720604,"scheduling_latency_s":0.720643,"redis_calls":13,"redis_duration_s":0.002999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03016,"completed_at":"2026-11-07T11:03:43.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.898547] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2597dbd7e3901cbef6600b98","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2597dbd7e3901cbef6600b98: start","job_status":"start","queue_duration_s":0.753921,"scheduling_latency_s":0.753966} vm-test-run-nixbot-gitlab> gitlab # [ 308.911487] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2597dbd7e3901cbef6600b98","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2597dbd7e3901cbef6600b98: done: 0.02675 sec","job_status":"done","queue_duration_s":0.753921,"scheduling_latency_s":0.753966,"redis_calls":13,"redis_duration_s":0.002865,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02675,"completed_at":"2026-11-07T11:03:43.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 308.952414] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45258105c9e463b35f91e486","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45258105c9e463b35f91e486: start","job_status":"start","queue_duration_s":0.783805,"scheduling_latency_s":0.783846} vm-test-run-nixbot-gitlab> gitlab # [ 308.966408] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45258105c9e463b35f91e486","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45258105c9e463b35f91e486: done: 0.025568 sec","job_status":"done","queue_duration_s":0.783805,"scheduling_latency_s":0.783846,"redis_calls":13,"redis_duration_s":0.003727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025568,"completed_at":"2026-11-07T11:03:43.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.007346] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"299fbd979beed7f83c16f5c0","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-299fbd979beed7f83c16f5c0: start","job_status":"start","queue_duration_s":0.811091,"scheduling_latency_s":0.811122} vm-test-run-nixbot-gitlab> gitlab # [ 309.020417] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"299fbd979beed7f83c16f5c0","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-299fbd979beed7f83c16f5c0: done: 0.02151 sec","job_status":"done","queue_duration_s":0.811091,"scheduling_latency_s":0.811122,"redis_calls":13,"redis_duration_s":0.002386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001569,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02151,"completed_at":"2026-11-07T11:03:43.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.060749] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9cf93641efcf3951663609","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9cf93641efcf3951663609: start","job_status":"start","queue_duration_s":0.834403,"scheduling_latency_s":0.834434} vm-test-run-nixbot-gitlab> gitlab # [ 309.073430] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9cf93641efcf3951663609","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9cf93641efcf3951663609: done: 0.019629 sec","job_status":"done","queue_duration_s":0.834403,"scheduling_latency_s":0.834434,"redis_calls":13,"redis_duration_s":0.002358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019629,"completed_at":"2026-11-07T11:03:43.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.113664] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9fb9409476101d24069186a","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9fb9409476101d24069186a: start","job_status":"start","queue_duration_s":0.855613,"scheduling_latency_s":0.855641} vm-test-run-nixbot-gitlab> gitlab # [ 309.126410] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9fb9409476101d24069186a","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9fb9409476101d24069186a: done: 0.022091 sec","job_status":"done","queue_duration_s":0.855613,"scheduling_latency_s":0.855641,"redis_calls":13,"redis_duration_s":0.002794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022091,"completed_at":"2026-11-07T11:03:43.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.167893] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21fac28bd9c33632e655085","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21fac28bd9c33632e655085: start","job_status":"start","queue_duration_s":0.879311,"scheduling_latency_s":0.879338} vm-test-run-nixbot-gitlab> gitlab # [ 309.180960] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21fac28bd9c33632e655085","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21fac28bd9c33632e655085: done: 0.020042 sec","job_status":"done","queue_duration_s":0.879311,"scheduling_latency_s":0.879338,"redis_calls":13,"redis_duration_s":0.002309,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020042,"completed_at":"2026-11-07T11:03:43.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.221147] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d261ee2145b4c6cf35c8dbf1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d261ee2145b4c6cf35c8dbf1: start","job_status":"start","queue_duration_s":0.901032,"scheduling_latency_s":0.901059} vm-test-run-nixbot-gitlab> gitlab # [ 309.233780] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d261ee2145b4c6cf35c8dbf1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d261ee2145b4c6cf35c8dbf1: done: 0.020804 sec","job_status":"done","queue_duration_s":0.901032,"scheduling_latency_s":0.901059,"redis_calls":13,"redis_duration_s":0.002312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001521,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020804,"completed_at":"2026-11-07T11:03:43.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.274161] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f97e740e6715288e955f4009","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f97e740e6715288e955f4009: start","job_status":"start","queue_duration_s":0.923704,"scheduling_latency_s":0.923735} vm-test-run-nixbot-gitlab> gitlab # [ 309.287028] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f97e740e6715288e955f4009","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f97e740e6715288e955f4009: done: 0.023369 sec","job_status":"done","queue_duration_s":0.923704,"scheduling_latency_s":0.923735,"redis_calls":13,"redis_duration_s":0.00454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023369,"completed_at":"2026-11-07T11:03:43.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.327494] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5db6f43b1b99472a448a97f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5db6f43b1b99472a448a97f: start","job_status":"start","queue_duration_s":0.948748,"scheduling_latency_s":0.948776} vm-test-run-nixbot-gitlab> gitlab # [ 309.340370] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5db6f43b1b99472a448a97f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5db6f43b1b99472a448a97f: done: 0.022469 sec","job_status":"done","queue_duration_s":0.948748,"scheduling_latency_s":0.948776,"redis_calls":13,"redis_duration_s":0.002409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001564,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022469,"completed_at":"2026-11-07T11:03:43.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.381907] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac6f5111576b16f1059b48e3","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac6f5111576b16f1059b48e3: start","job_status":"start","queue_duration_s":0.972991,"scheduling_latency_s":0.97302} vm-test-run-nixbot-gitlab> gitlab # [ 309.394664] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac6f5111576b16f1059b48e3","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac6f5111576b16f1059b48e3: done: 0.02008 sec","job_status":"done","queue_duration_s":0.972991,"scheduling_latency_s":0.97302,"redis_calls":13,"redis_duration_s":0.002322,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02008,"completed_at":"2026-11-07T11:03:43.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.434976] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67814f798b29e32b5579df22","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67814f798b29e32b5579df22: start","job_status":"start","queue_duration_s":0.994629,"scheduling_latency_s":0.994655} vm-test-run-nixbot-gitlab> gitlab # [ 309.447851] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67814f798b29e32b5579df22","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67814f798b29e32b5579df22: done: 0.020707 sec","job_status":"done","queue_duration_s":0.994629,"scheduling_latency_s":0.994655,"redis_calls":13,"redis_duration_s":0.002861,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020707,"completed_at":"2026-11-07T11:03:43.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.488599] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500d49e72236a943b04cd32f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500d49e72236a943b04cd32f: start","job_status":"start","queue_duration_s":1.017146,"scheduling_latency_s":1.017178} vm-test-run-nixbot-gitlab> gitlab # [ 309.501358] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500d49e72236a943b04cd32f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500d49e72236a943b04cd32f: done: 0.021588 sec","job_status":"done","queue_duration_s":1.017146,"scheduling_latency_s":1.017178,"redis_calls":13,"redis_duration_s":0.002542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021588,"completed_at":"2026-11-07T11:03:43.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.541700] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b9c4a29c79328a3dad674ce","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b9c4a29c79328a3dad674ce: start","job_status":"start","queue_duration_s":1.040432,"scheduling_latency_s":1.040461} vm-test-run-nixbot-gitlab> gitlab # [ 309.554593] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b9c4a29c79328a3dad674ce","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b9c4a29c79328a3dad674ce: done: 0.021793 sec","job_status":"done","queue_duration_s":1.040432,"scheduling_latency_s":1.040461,"redis_calls":13,"redis_duration_s":0.002438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001619,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021793,"completed_at":"2026-11-07T11:03:43.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.596032] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"443b93f2758cabe2bfdd3d9f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-443b93f2758cabe2bfdd3d9f: start","job_status":"start","queue_duration_s":1.064025,"scheduling_latency_s":1.064051} vm-test-run-nixbot-gitlab> gitlab # [ 309.608645] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"443b93f2758cabe2bfdd3d9f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-443b93f2758cabe2bfdd3d9f: done: 0.022745 sec","job_status":"done","queue_duration_s":1.064025,"scheduling_latency_s":1.064051,"redis_calls":13,"redis_duration_s":0.002502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001663,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022745,"completed_at":"2026-11-07T11:03:43.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.649122] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d3c1cf18ab31d8cf6e66f42","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d3c1cf18ab31d8cf6e66f42: start","job_status":"start","queue_duration_s":1.088657,"scheduling_latency_s":1.088684} vm-test-run-nixbot-gitlab> gitlab # [ 309.661796] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d3c1cf18ab31d8cf6e66f42","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d3c1cf18ab31d8cf6e66f42: done: 0.022496 sec","job_status":"done","queue_duration_s":1.088657,"scheduling_latency_s":1.088684,"redis_calls":13,"redis_duration_s":0.0025399999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022496,"completed_at":"2026-11-07T11:03:43.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.702475] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f5a7e59c53a5810736886c1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f5a7e59c53a5810736886c1: start","job_status":"start","queue_duration_s":1.112796,"scheduling_latency_s":1.112823} vm-test-run-nixbot-gitlab> gitlab # [ 309.715345] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f5a7e59c53a5810736886c1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f5a7e59c53a5810736886c1: done: 0.020347 sec","job_status":"done","queue_duration_s":1.112796,"scheduling_latency_s":1.112823,"redis_calls":13,"redis_duration_s":0.002526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020347,"completed_at":"2026-11-07T11:03:43.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.756257] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0abb407867bb8bbbbe6eb29","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0abb407867bb8bbbbe6eb29: start","job_status":"start","queue_duration_s":1.135978,"scheduling_latency_s":1.136007} vm-test-run-nixbot-gitlab> gitlab # [ 309.769078] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0abb407867bb8bbbbe6eb29","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0abb407867bb8bbbbe6eb29: done: 0.021176 sec","job_status":"done","queue_duration_s":1.135978,"scheduling_latency_s":1.136007,"redis_calls":13,"redis_duration_s":0.002473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001624,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021176,"completed_at":"2026-11-07T11:03:43.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.810619] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d1ed95a6d7450cf0662f1cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d1ed95a6d7450cf0662f1cd: start","job_status":"start","queue_duration_s":1.158849,"scheduling_latency_s":1.158877} vm-test-run-nixbot-gitlab> gitlab # [ 309.823310] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d1ed95a6d7450cf0662f1cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d1ed95a6d7450cf0662f1cd: done: 0.020196 sec","job_status":"done","queue_duration_s":1.158849,"scheduling_latency_s":1.158877,"redis_calls":13,"redis_duration_s":0.002478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001641,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020196,"completed_at":"2026-11-07T11:03:43.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.864290] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d99865bf82b4e1af83c1f646","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d99865bf82b4e1af83c1f646: start","job_status":"start","queue_duration_s":1.180644,"scheduling_latency_s":1.180671} vm-test-run-nixbot-gitlab> gitlab # [ 309.877133] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d99865bf82b4e1af83c1f646","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d99865bf82b4e1af83c1f646: done: 0.02104 sec","job_status":"done","queue_duration_s":1.180644,"scheduling_latency_s":1.180671,"redis_calls":13,"redis_duration_s":0.00243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02104,"completed_at":"2026-11-07T11:03:43.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.917502] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc3803f352f6b8e624891d6d","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc3803f352f6b8e624891d6d: start","job_status":"start","queue_duration_s":1.203271,"scheduling_latency_s":1.203296} vm-test-run-nixbot-gitlab> gitlab # [ 309.930342] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc3803f352f6b8e624891d6d","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc3803f352f6b8e624891d6d: done: 0.018943 sec","job_status":"done","queue_duration_s":1.203271,"scheduling_latency_s":1.203296,"redis_calls":13,"redis_duration_s":0.002356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001544,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018943,"completed_at":"2026-11-07T11:03:43.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 309.970735] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a7e4fe934ac763249624f1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a7e4fe934ac763249624f1: start","job_status":"start","queue_duration_s":1.223752,"scheduling_latency_s":1.223777} vm-test-run-nixbot-gitlab> gitlab # [ 309.984704] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a7e4fe934ac763249624f1","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a7e4fe934ac763249624f1: done: 0.02192 sec","job_status":"done","queue_duration_s":1.223752,"scheduling_latency_s":1.223777,"redis_calls":13,"redis_duration_s":0.0026550000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02192,"completed_at":"2026-11-07T11:03:43.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.025277] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:43.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55099a1abeddd2b5052cbb56","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55099a1abeddd2b5052cbb56: start","job_status":"start","queue_duration_s":1.247323,"scheduling_latency_s":1.247349} vm-test-run-nixbot-gitlab> gitlab # [ 310.038072] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55099a1abeddd2b5052cbb56","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55099a1abeddd2b5052cbb56: done: 0.020278 sec","job_status":"done","queue_duration_s":1.247323,"scheduling_latency_s":1.247349,"redis_calls":13,"redis_duration_s":0.002376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020278,"completed_at":"2026-11-07T11:03:44.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.078761] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aad74e9c4b928ead2db1ce4","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aad74e9c4b928ead2db1ce4: start","job_status":"start","queue_duration_s":1.269393,"scheduling_latency_s":1.269422} vm-test-run-nixbot-gitlab> gitlab # [ 310.091761] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aad74e9c4b928ead2db1ce4","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aad74e9c4b928ead2db1ce4: done: 0.019681 sec","job_status":"done","queue_duration_s":1.269393,"scheduling_latency_s":1.269422,"redis_calls":13,"redis_duration_s":0.002386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019681,"completed_at":"2026-11-07T11:03:44.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.132854] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de3595e8e77d6524ae44e84","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de3595e8e77d6524ae44e84: start","job_status":"start","queue_duration_s":1.290669,"scheduling_latency_s":1.290698} vm-test-run-nixbot-gitlab> gitlab # [ 310.145679] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de3595e8e77d6524ae44e84","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de3595e8e77d6524ae44e84: done: 0.019324 sec","job_status":"done","queue_duration_s":1.290669,"scheduling_latency_s":1.290698,"redis_calls":13,"redis_duration_s":0.002294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019324,"completed_at":"2026-11-07T11:03:44.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.187121] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38228a3133c945e659f96dc3","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38228a3133c945e659f96dc3: start","job_status":"start","queue_duration_s":1.311578,"scheduling_latency_s":1.311605} vm-test-run-nixbot-gitlab> gitlab # [ 310.200150] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38228a3133c945e659f96dc3","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38228a3133c945e659f96dc3: done: 0.021342 sec","job_status":"done","queue_duration_s":1.311578,"scheduling_latency_s":1.311605,"redis_calls":13,"redis_duration_s":0.002482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001574,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021342,"completed_at":"2026-11-07T11:03:44.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.240842] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaa82aaafa3e32ec4a1c7f6f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaa82aaafa3e32ec4a1c7f6f: start","job_status":"start","queue_duration_s":1.33459,"scheduling_latency_s":1.334618} vm-test-run-nixbot-gitlab> gitlab # [ 310.253699] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaa82aaafa3e32ec4a1c7f6f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaa82aaafa3e32ec4a1c7f6f: done: 0.02022 sec","job_status":"done","queue_duration_s":1.33459,"scheduling_latency_s":1.334618,"redis_calls":13,"redis_duration_s":0.002473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001617,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02022,"completed_at":"2026-11-07T11:03:44.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.294477] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9139aec26afa8eaeefc69d12","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9139aec26afa8eaeefc69d12: start","job_status":"start","queue_duration_s":1.356441,"scheduling_latency_s":1.356469} vm-test-run-nixbot-gitlab> gitlab # [ 310.307453] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9139aec26afa8eaeefc69d12","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9139aec26afa8eaeefc69d12: done: 0.018572 sec","job_status":"done","queue_duration_s":1.356441,"scheduling_latency_s":1.356469,"redis_calls":13,"redis_duration_s":0.002187,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018572,"completed_at":"2026-11-07T11:03:44.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.348475] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a6e35470ec4fe54aab3b7be","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a6e35470ec4fe54aab3b7be: start","job_status":"start","queue_duration_s":1.376611,"scheduling_latency_s":1.376639} vm-test-run-nixbot-gitlab> gitlab # [ 310.361328] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a6e35470ec4fe54aab3b7be","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a6e35470ec4fe54aab3b7be: done: 0.0189 sec","job_status":"done","queue_duration_s":1.376611,"scheduling_latency_s":1.376639,"redis_calls":13,"redis_duration_s":0.0023150000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0189,"completed_at":"2026-11-07T11:03:44.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.403036] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a403854aa9bc72fb04c5510","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a403854aa9bc72fb04c5510: start","job_status":"start","queue_duration_s":1.397542,"scheduling_latency_s":1.397568} vm-test-run-nixbot-gitlab> gitlab # [ 310.415977] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a403854aa9bc72fb04c5510","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a403854aa9bc72fb04c5510: done: 0.020324 sec","job_status":"done","queue_duration_s":1.397542,"scheduling_latency_s":1.397568,"redis_calls":13,"redis_duration_s":0.002508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001568,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020324,"completed_at":"2026-11-07T11:03:44.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.457287] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5a67500306ed54ac5e0eef2","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5a67500306ed54ac5e0eef2: start","job_status":"start","queue_duration_s":1.419472,"scheduling_latency_s":1.419501} vm-test-run-nixbot-gitlab> gitlab # [ 310.470156] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5a67500306ed54ac5e0eef2","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5a67500306ed54ac5e0eef2: done: 0.019408 sec","job_status":"done","queue_duration_s":1.419472,"scheduling_latency_s":1.419501,"redis_calls":13,"redis_duration_s":0.002451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001606,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019408,"completed_at":"2026-11-07T11:03:44.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.510695] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84a60f8ca48257ad933d4890","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84a60f8ca48257ad933d4890: start","job_status":"start","queue_duration_s":1.44052,"scheduling_latency_s":1.440576} vm-test-run-nixbot-gitlab> gitlab # [ 310.523405] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84a60f8ca48257ad933d4890","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84a60f8ca48257ad933d4890: done: 0.019856 sec","job_status":"done","queue_duration_s":1.44052,"scheduling_latency_s":1.440576,"redis_calls":13,"redis_duration_s":0.002528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019856,"completed_at":"2026-11-07T11:03:44.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.564301] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a453cc744432c8e830ed4efa","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a453cc744432c8e830ed4efa: start","job_status":"start","queue_duration_s":1.462124,"scheduling_latency_s":1.462152} vm-test-run-nixbot-gitlab> gitlab # [ 310.577095] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a453cc744432c8e830ed4efa","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a453cc744432c8e830ed4efa: done: 0.018808 sec","job_status":"done","queue_duration_s":1.462124,"scheduling_latency_s":1.462152,"redis_calls":13,"redis_duration_s":0.0021219999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018808,"completed_at":"2026-11-07T11:03:44.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.618765] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47f39ca5207857b16b84be71","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47f39ca5207857b16b84be71: start","job_status":"start","queue_duration_s":1.482837,"scheduling_latency_s":1.482865} vm-test-run-nixbot-gitlab> gitlab # [ 310.631722] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47f39ca5207857b16b84be71","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47f39ca5207857b16b84be71: done: 0.018514 sec","job_status":"done","queue_duration_s":1.482837,"scheduling_latency_s":1.482865,"redis_calls":13,"redis_duration_s":0.002377,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001587,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018514,"completed_at":"2026-11-07T11:03:44.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.672302] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cb0d7566981b21bce51bc5f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cb0d7566981b21bce51bc5f: start","job_status":"start","queue_duration_s":1.50383,"scheduling_latency_s":1.503859} vm-test-run-nixbot-gitlab> gitlab # [ 310.685032] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cb0d7566981b21bce51bc5f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cb0d7566981b21bce51bc5f: done: 0.019424 sec","job_status":"done","queue_duration_s":1.50383,"scheduling_latency_s":1.503859,"redis_calls":13,"redis_duration_s":0.002402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001551,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019424,"completed_at":"2026-11-07T11:03:44.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.725419] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec11ed5322c53bf611b020a4","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec11ed5322c53bf611b020a4: start","job_status":"start","queue_duration_s":1.525131,"scheduling_latency_s":1.525159} vm-test-run-nixbot-gitlab> gitlab # [ 310.738237] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec11ed5322c53bf611b020a4","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec11ed5322c53bf611b020a4: done: 0.018501 sec","job_status":"done","queue_duration_s":1.525131,"scheduling_latency_s":1.525159,"redis_calls":13,"redis_duration_s":0.002334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018501,"completed_at":"2026-11-07T11:03:44.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.779278] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"741769c4bd2e7e951e492738","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-741769c4bd2e7e951e492738: start","job_status":"start","queue_duration_s":1.545926,"scheduling_latency_s":1.545954} vm-test-run-nixbot-gitlab> gitlab # [ 310.793492] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"741769c4bd2e7e951e492738","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-741769c4bd2e7e951e492738: done: 0.019849 sec","job_status":"done","queue_duration_s":1.545926,"scheduling_latency_s":1.545954,"redis_calls":13,"redis_duration_s":0.00246,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019849,"completed_at":"2026-11-07T11:03:44.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.833769] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90026b19c69dd4fa81ea6c36","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90026b19c69dd4fa81ea6c36: start","job_status":"start","queue_duration_s":1.567641,"scheduling_latency_s":1.567668} vm-test-run-nixbot-gitlab> gitlab # [ 310.846548] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90026b19c69dd4fa81ea6c36","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90026b19c69dd4fa81ea6c36: done: 0.020344 sec","job_status":"done","queue_duration_s":1.567641,"scheduling_latency_s":1.567668,"redis_calls":13,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020344,"completed_at":"2026-11-07T11:03:44.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.887691] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86b5c4ff0e9f875aecadc0cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86b5c4ff0e9f875aecadc0cd: start","job_status":"start","queue_duration_s":1.589684,"scheduling_latency_s":1.589713} vm-test-run-nixbot-gitlab> gitlab # [ 310.900712] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86b5c4ff0e9f875aecadc0cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86b5c4ff0e9f875aecadc0cd: done: 0.019577 sec","job_status":"done","queue_duration_s":1.589684,"scheduling_latency_s":1.589713,"redis_calls":13,"redis_duration_s":0.002314,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001443,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019577,"completed_at":"2026-11-07T11:03:44.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.940922] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27048a174d48d20f742a59c","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27048a174d48d20f742a59c: start","job_status":"start","queue_duration_s":1.611472,"scheduling_latency_s":1.611498} vm-test-run-nixbot-gitlab> gitlab # [ 310.953751] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27048a174d48d20f742a59c","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27048a174d48d20f742a59c: done: 0.019278 sec","job_status":"done","queue_duration_s":1.611472,"scheduling_latency_s":1.611498,"redis_calls":13,"redis_duration_s":0.002426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019278,"completed_at":"2026-11-07T11:03:44.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 310.994233] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff0c1a55e48d673187bea2cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff0c1a55e48d673187bea2cd: start","job_status":"start","queue_duration_s":1.633326,"scheduling_latency_s":1.633355} vm-test-run-nixbot-gitlab> gitlab # [ 311.008198] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff0c1a55e48d673187bea2cd","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff0c1a55e48d673187bea2cd: done: 0.02051 sec","job_status":"done","queue_duration_s":1.633326,"scheduling_latency_s":1.633355,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001506,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02051,"completed_at":"2026-11-07T11:03:44.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.048586] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a955bd3bc84dfb805f716c9f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a955bd3bc84dfb805f716c9f: start","job_status":"start","queue_duration_s":1.655565,"scheduling_latency_s":1.655595} vm-test-run-nixbot-gitlab> gitlab # [ 311.061438] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a955bd3bc84dfb805f716c9f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a955bd3bc84dfb805f716c9f: done: 0.028306 sec","job_status":"done","queue_duration_s":1.655565,"scheduling_latency_s":1.655595,"redis_calls":13,"redis_duration_s":0.008569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004185,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004384,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028306,"completed_at":"2026-11-07T11:03:44.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.101839] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff9e798a9d25f3a4bdc2d113","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff9e798a9d25f3a4bdc2d113: start","job_status":"start","queue_duration_s":1.689123,"scheduling_latency_s":1.689162} vm-test-run-nixbot-gitlab> gitlab # [ 311.114634] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff9e798a9d25f3a4bdc2d113","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff9e798a9d25f3a4bdc2d113: done: 0.024798 sec","job_status":"done","queue_duration_s":1.689123,"scheduling_latency_s":1.689162,"redis_calls":13,"redis_duration_s":0.004379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024798,"completed_at":"2026-11-07T11:03:44.457Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.155105] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e792f407ad0e84d634261d","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e792f407ad0e84d634261d: start","job_status":"start","queue_duration_s":1.715452,"scheduling_latency_s":1.715482} vm-test-run-nixbot-gitlab> gitlab # [ 311.167853] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e792f407ad0e84d634261d","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e792f407ad0e84d634261d: done: 0.019944 sec","job_status":"done","queue_duration_s":1.715452,"scheduling_latency_s":1.715482,"redis_calls":13,"redis_duration_s":0.002347,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019944,"completed_at":"2026-11-07T11:03:44.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.210106] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b5f0b33ea66d4ee2b768ac","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b5f0b33ea66d4ee2b768ac: start","job_status":"start","queue_duration_s":1.738137,"scheduling_latency_s":1.738169} vm-test-run-nixbot-gitlab> gitlab # [ 311.222882] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b5f0b33ea66d4ee2b768ac","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b5f0b33ea66d4ee2b768ac: done: 0.021633 sec","job_status":"done","queue_duration_s":1.738137,"scheduling_latency_s":1.738169,"redis_calls":13,"redis_duration_s":0.002748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001729,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021633,"completed_at":"2026-11-07T11:03:44.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.263316] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4a155251a75185f2d12911f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4a155251a75185f2d12911f: start","job_status":"start","queue_duration_s":1.762138,"scheduling_latency_s":1.762194} vm-test-run-nixbot-gitlab> gitlab # [ 311.276261] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4a155251a75185f2d12911f","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4a155251a75185f2d12911f: done: 0.020668 sec","job_status":"done","queue_duration_s":1.762138,"scheduling_latency_s":1.762194,"redis_calls":13,"redis_duration_s":0.0023680000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020668,"completed_at":"2026-11-07T11:03:44.526Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.316895] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"784f2435b9426c7609ce5258","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-784f2435b9426c7609ce5258: start","job_status":"start","queue_duration_s":1.784395,"scheduling_latency_s":1.784425} vm-test-run-nixbot-gitlab> gitlab # [ 311.329632] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"784f2435b9426c7609ce5258","created_at":"2026-11-07T11:03:42.604Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-382c944029b94a70","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3324083d561d7c0a8445032fa769a4fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:42.743Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-784f2435b9426c7609ce5258: done: 0.020384 sec","job_status":"done","queue_duration_s":1.784395,"scheduling_latency_s":1.784425,"redis_calls":13,"redis_duration_s":0.002648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020384,"completed_at":"2026-11-07T11:03:44.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.369997] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.550Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"44beb1c1e8abc54f2db51b8c","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-44beb1c1e8abc54f2db51b8c: start","job_status":"start","queue_duration_s":1.211248,"scheduling_latency_s":1.211275} vm-test-run-nixbot-gitlab> gitlab # [ 311.383160] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.574Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 311.385029] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.592Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"44beb1c1e8abc54f2db51b8c","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-44beb1c1e8abc54f2db51b8c: done: 0.041718 sec","job_status":"done","queue_duration_s":1.211248,"scheduling_latency_s":1.211275,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003602,"duration_s":0.041718,"completed_at":"2026-11-07T11:03:44.592Z","load_balancing_strategy":"primary","db_duration_s":0.005483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.427184] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.593Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b41a332c40852e2d3ab25d6","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b41a332c40852e2d3ab25d6: start","job_status":"start","queue_duration_s":1.254658,"scheduling_latency_s":1.254686} vm-test-run-nixbot-gitlab> gitlab # [ 311.440096] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.608Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 311.442131] redis[788]: 10 changes in 300 seconds. Saving... vm-test-run-nixbot-gitlab> gitlab # [ 311.443149] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.620Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b41a332c40852e2d3ab25d6","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b41a332c40852e2d3ab25d6: done: 0.02673 sec","job_status":"done","queue_duration_s":1.254658,"scheduling_latency_s":1.254686,"redis_calls":10,"redis_duration_s":0.002341,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001387,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02673,"completed_at":"2026-11-07T11:03:44.620Z","load_balancing_strategy":"primary","db_duration_s":0.002521,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.483181] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.622Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"690f4710d1c7ba94adceddfb","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-690f4710d1c7ba94adceddfb: start","job_status":"start","queue_duration_s":1.283046,"scheduling_latency_s":1.283074} vm-test-run-nixbot-gitlab> gitlab # [ 311.495973] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.637Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 311.497788] redis[788]: Background saving started by pid 3621 vm-test-run-nixbot-gitlab> gitlab # [ 311.498752] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.649Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"690f4710d1c7ba94adceddfb","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-690f4710d1c7ba94adceddfb: done: 0.027517 sec","job_status":"done","queue_duration_s":1.283046,"scheduling_latency_s":1.283074,"redis_calls":10,"redis_duration_s":0.002378,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001324,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027517,"completed_at":"2026-11-07T11:03:44.649Z","load_balancing_strategy":"primary","db_duration_s":0.002389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.538515] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.652Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"58206cd75f04a52d46755661","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-58206cd75f04a52d46755661: start","job_status":"start","queue_duration_s":1.313375,"scheduling_latency_s":1.313408} vm-test-run-nixbot-gitlab> gitlab # [ 311.551215] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.669Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 311.553142] redis[3621]: BGSAVE done, 409 keys saved, 0 keys skipped, 110183 bytes written. vm-test-run-nixbot-gitlab> gitlab # [ 311.554419] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:03:44.682Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"58206cd75f04a52d46755661","created_at":"2026-11-07T11:03:43.320Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-0e4d6ebcf5dc4268","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"23c751d00f0f4ead67738186da68ba2a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:03:43.339Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-58206cd75f04a52d46755661: done: 0.029539 sec","job_status":"done","queue_duration_s":1.313375,"scheduling_latency_s":1.313408,"redis_calls":10,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001366,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029539,"completed_at":"2026-11-07T11:03:44.682Z","load_balancing_strategy":"primary","db_duration_s":0.002116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 311.594463] redis[3621]: DB saved on disk vm-test-run-nixbot-gitlab> gitlab # [ 311.595188] redis[3621]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB vm-test-run-nixbot-gitlab> gitlab # [ 311.596394] redis[788]: Background saving terminated with success vm-test-run-nixbot-gitlab> gitlab # [ 312.581817] postgres[861]: [861] LOG: checkpoint starting: time vm-test-run-nixbot-gitlab> gitlab # [ 314.552263] gitaly[928]: time="2026-11-07T11:03:49.275Z" level=info msg="burden monitor stats" pid=928 vm-test-run-nixbot-gitlab> gitlab # [ 328.075395] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:02.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f7f7ce866d55e5ca04945d5f","created_at":"2026-11-07T11:04:02.788Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1c4bcfb298c3fc0ceb419baf5f40ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:04:02.795Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f7f7ce866d55e5ca04945d5f: start","job_status":"start","queue_duration_s":0.002547,"scheduling_latency_s":0.002594} vm-test-run-nixbot-gitlab> gitlab # [ 328.165917] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:02.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f7f7ce866d55e5ca04945d5f","created_at":"2026-11-07T11:04:02.788Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1c4bcfb298c3fc0ceb419baf5f40ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:04:02.795Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f7f7ce866d55e5ca04945d5f: done: 0.090704 sec","job_status":"done","queue_duration_s":0.002547,"scheduling_latency_s":0.002594,"redis_calls":7,"redis_duration_s":0.027970000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090704,"completed_at":"2026-11-07T11:04:02.888Z","load_balancing_strategy":"primary","db_duration_s":0.010712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.226114] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:02.903Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"52cb487ba9051128299e2af5","created_at":"2026-11-07T11:04:02.822Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d10c0dfcbbf5f409b0606d44ef3cc5f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.837Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-52cb487ba9051128299e2af5: start","job_status":"start","queue_duration_s":0.066405,"scheduling_latency_s":0.066459} vm-test-run-nixbot-gitlab> gitlab # [ 328.345211] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.068Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"52cb487ba9051128299e2af5","created_at":"2026-11-07T11:04:02.822Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d10c0dfcbbf5f409b0606d44ef3cc5f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.837Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-52cb487ba9051128299e2af5: done: 0.164352 sec","job_status":"done","queue_duration_s":0.066405,"scheduling_latency_s":0.066459,"redis_calls":7,"redis_duration_s":0.054262000000000005,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.045961,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.164352,"completed_at":"2026-11-07T11:04:03.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.407231] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2773cc4cd8cfac964853542b","created_at":"2026-11-07T11:04:02.866Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24be74d4b09b43c5cf081a3b1f2a8395","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.875Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2773cc4cd8cfac964853542b: start","job_status":"start","queue_duration_s":0.216957,"scheduling_latency_s":0.217009} vm-test-run-nixbot-gitlab> gitlab # [ 328.584032] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.306Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2773cc4cd8cfac964853542b","created_at":"2026-11-07T11:04:02.866Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24be74d4b09b43c5cf081a3b1f2a8395","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.875Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2773cc4cd8cfac964853542b: done: 0.214201 sec","job_status":"done","queue_duration_s":0.216957,"scheduling_latency_s":0.217009,"redis_calls":9,"redis_duration_s":0.019839,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.011918,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00502824,"exclusive_lock_hold_duration_s":0.04626906,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.214201,"completed_at":"2026-11-07T11:04:03.306Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 328.650291] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"42b3583fc235f3468c621b29","created_at":"2026-11-07T11:04:02.894Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddaae991d47b5a9eb31852906c3c1ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.902Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-42b3583fc235f3468c621b29: start","job_status":"start","queue_duration_s":0.437212,"scheduling_latency_s":0.437268} vm-test-run-nixbot-gitlab> gitlab # [ 328.765934] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.489Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 328.945004] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"42b3583fc235f3468c621b29","created_at":"2026-11-07T11:04:02.894Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddaae991d47b5a9eb31852906c3c1ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.902Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-42b3583fc235f3468c621b29: done: 0.327534 sec","job_status":"done","queue_duration_s":0.437212,"scheduling_latency_s":0.437268,"redis_calls":7,"redis_duration_s":0.010123,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.327534,"completed_at":"2026-11-07T11:04:03.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.012205] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"baa2f05307df812cd457d8e6","created_at":"2026-11-07T11:04:02.963Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50d2edc0986b6d2da1a7c2e2d9b1ccb1","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.991Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-baa2f05307df812cd457d8e6: start","job_status":"start","queue_duration_s":0.684975,"scheduling_latency_s":0.685027} vm-test-run-nixbot-gitlab> gitlab # [ 329.045815] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.768Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"baa2f05307df812cd457d8e6","created_at":"2026-11-07T11:04:02.963Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50d2edc0986b6d2da1a7c2e2d9b1ccb1","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:02.991Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-baa2f05307df812cd457d8e6: done: 0.091979 sec","job_status":"done","queue_duration_s":0.684975,"scheduling_latency_s":0.685027,"redis_calls":12,"redis_duration_s":0.021519,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001228,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005969,"duration_s":0.091979,"completed_at":"2026-11-07T11:04:03.768Z","load_balancing_strategy":"primary","db_duration_s":0.014896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.110051] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6e10cadccc94519e962dad2a","created_at":"2026-11-07T11:04:03.007Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cccc5eaa33ff6876d9fbcc638c4550a3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.012Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6e10cadccc94519e962dad2a: start","job_status":"start","queue_duration_s":0.759701,"scheduling_latency_s":0.759747} vm-test-run-nixbot-gitlab> gitlab # [ 329.127186] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6e10cadccc94519e962dad2a","created_at":"2026-11-07T11:04:03.007Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cccc5eaa33ff6876d9fbcc638c4550a3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.012Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6e10cadccc94519e962dad2a: done: 0.053227 sec","job_status":"done","queue_duration_s":0.759701,"scheduling_latency_s":0.759747,"redis_calls":7,"redis_duration_s":0.004368,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002326,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.053227,"completed_at":"2026-11-07T11:04:03.825Z","load_balancing_strategy":"primary","db_duration_s":0.013775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.173257] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03f92e200e47ae05d2d60665","created_at":"2026-11-07T11:04:03.020Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bf3f75a1127ba911c13126827034902","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.026Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03f92e200e47ae05d2d60665: start","job_status":"start","queue_duration_s":0.802377,"scheduling_latency_s":0.802425} vm-test-run-nixbot-gitlab> gitlab # [ 329.189398] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.871Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 329.192692] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03f92e200e47ae05d2d60665","created_at":"2026-11-07T11:04:03.020Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bf3f75a1127ba911c13126827034902","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.026Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03f92e200e47ae05d2d60665: done: 0.060891 sec","job_status":"done","queue_duration_s":0.802377,"scheduling_latency_s":0.802425,"redis_calls":7,"redis_duration_s":0.003853,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060891,"completed_at":"2026-11-07T11:04:03.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.234661] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"14c6680ee0179a5dc45a05f4","created_at":"2026-11-07T11:04:03.044Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"913231002b3629704ff159450b5bed14","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:04:03.053Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-14c6680ee0179a5dc45a05f4: start","job_status":"start","queue_duration_s":0.840551,"scheduling_latency_s":0.840596} vm-test-run-nixbot-gitlab> gitlab # [ 329.248808] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.916Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 329.252621] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"14c6680ee0179a5dc45a05f4","created_at":"2026-11-07T11:04:03.044Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"913231002b3629704ff159450b5bed14","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:04:03.053Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-14c6680ee0179a5dc45a05f4: done: 0.035183 sec","job_status":"done","queue_duration_s":0.840551,"scheduling_latency_s":0.840596,"redis_calls":7,"redis_duration_s":0.00334,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035183,"completed_at":"2026-11-07T11:04:03.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.293881] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:03.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c6eb5b6f40fa87983e120968","created_at":"2026-11-07T11:04:03.069Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af5514d3d2ee5d639bf3d98f84219a02","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.078Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c6eb5b6f40fa87983e120968: start","job_status":"start","queue_duration_s":0.853292,"scheduling_latency_s":0.853333} vm-test-run-nixbot-gitlab> gitlab # [ 329.963280] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c6eb5b6f40fa87983e120968","created_at":"2026-11-07T11:04:03.069Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af5514d3d2ee5d639bf3d98f84219a02","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.078Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c6eb5b6f40fa87983e120968: done: 0.754355 sec","job_status":"done","queue_duration_s":0.853292,"scheduling_latency_s":0.853333,"redis_calls":562,"redis_duration_s":0.178037,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.176797,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.700345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.754355,"completed_at":"2026-11-07T11:04:04.686Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 329.965991] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"3415324495e0b9fb8a977b80","created_at":"2026-11-07T11:04:03.091Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db18e8afb66ef0abece01b796a9f0387","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.107Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-3415324495e0b9fb8a977b80: start","job_status":"start","queue_duration_s":1.582135,"scheduling_latency_s":1.58218} vm-test-run-nixbot-gitlab> gitlab # [ 330.149604] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"3415324495e0b9fb8a977b80","created_at":"2026-11-07T11:04:03.091Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db18e8afb66ef0abece01b796a9f0387","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.107Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-3415324495e0b9fb8a977b80: done: 0.182581 sec","job_status":"done","queue_duration_s":1.582135,"scheduling_latency_s":1.58218,"redis_calls":12,"redis_duration_s":0.004605,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00089,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00133638,"exclusive_lock_hold_duration_s":0.013197140000000001,"duration_s":0.182581,"completed_at":"2026-11-07T11:04:04.872Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.216328] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.874Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"fa0bcd4f101858366915a57d","created_at":"2026-11-07T11:04:03.123Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1904a18b31af29338384aab7d7deb1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:04:03.128Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-fa0bcd4f101858366915a57d: start","job_status":"start","queue_duration_s":1.746573,"scheduling_latency_s":1.746615} vm-test-run-nixbot-gitlab> gitlab # [ 330.234772] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.889Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 330.239212] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"fa0bcd4f101858366915a57d","created_at":"2026-11-07T11:04:03.123Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1904a18b31af29338384aab7d7deb1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:04:03.128Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-fa0bcd4f101858366915a57d: done: 0.028505 sec","job_status":"done","queue_duration_s":1.746573,"scheduling_latency_s":1.746615,"redis_calls":7,"redis_duration_s":0.002204,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000685,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028505,"completed_at":"2026-11-07T11:04:04.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.285895] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"09049d39da13272c4c7026ef","created_at":"2026-11-07T11:04:03.141Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55717344e8ae476ff2bfc9134a1f68ed","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.162Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-09049d39da13272c4c7026ef: start","job_status":"start","queue_duration_s":1.742824,"scheduling_latency_s":1.742863} vm-test-run-nixbot-gitlab> gitlab # [ 330.301065] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"09049d39da13272c4c7026ef","created_at":"2026-11-07T11:04:03.141Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55717344e8ae476ff2bfc9134a1f68ed","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.162Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-09049d39da13272c4c7026ef: done: 0.062589 sec","job_status":"done","queue_duration_s":1.742824,"scheduling_latency_s":1.742863,"redis_calls":10,"redis_duration_s":0.0034289999999999998,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000544,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001942,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069214,"duration_s":0.062589,"completed_at":"2026-11-07T11:04:04.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.348469] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:04.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"99bacf7aa93b7d6b1fac57cb","created_at":"2026-11-07T11:04:03.212Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b11c01ed01c931a0b7570ed31af1bdf","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.222Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-99bacf7aa93b7d6b1fac57cb: start","job_status":"start","queue_duration_s":1.749155,"scheduling_latency_s":1.74936} vm-test-run-nixbot-gitlab> gitlab # [ 330.364353] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"99bacf7aa93b7d6b1fac57cb","created_at":"2026-11-07T11:04:03.212Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b11c01ed01c931a0b7570ed31af1bdf","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.222Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-99bacf7aa93b7d6b1fac57cb: done: 0.06598 sec","job_status":"done","queue_duration_s":1.749155,"scheduling_latency_s":1.74936,"redis_calls":10,"redis_duration_s":0.0035960000000000002,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002108,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0117359,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.06598,"completed_at":"2026-11-07T11:04:05.037Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004227,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 330.409080] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e92b6fe2a6cc489669c8c01d","created_at":"2026-11-07T11:04:03.234Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2855b8f28892c3d70e9408b87e7baa90","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.246Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e92b6fe2a6cc489669c8c01d: start","job_status":"start","queue_duration_s":1.79445,"scheduling_latency_s":1.794488} vm-test-run-nixbot-gitlab> gitlab # [ 330.423084] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e92b6fe2a6cc489669c8c01d","created_at":"2026-11-07T11:04:03.234Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2855b8f28892c3d70e9408b87e7baa90","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.246Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e92b6fe2a6cc489669c8c01d: done: 0.069594 sec","job_status":"done","queue_duration_s":1.79445,"scheduling_latency_s":1.794488,"redis_calls":9,"redis_duration_s":0.004346,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001659,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00106596,"exclusive_lock_hold_duration_s":0.00815658,"duration_s":0.069594,"completed_at":"2026-11-07T11:04:05.109Z","load_balancing_strategy":"primary","db_duration_s":0.003394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.466205] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.112Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"de8824b13815475a5ecc65a4","created_at":"2026-11-07T11:04:03.260Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"b65054f97e2c34aa20978c826b3c1d13","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.278Z","job_size_bytes":3,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-de8824b13815475a5ecc65a4: start","job_status":"start","queue_duration_s":1.834498,"scheduling_latency_s":1.834589} vm-test-run-nixbot-gitlab> gitlab # [ 330.480907] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.142Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"de8824b13815475a5ecc65a4","created_at":"2026-11-07T11:04:03.260Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"b65054f97e2c34aa20978c826b3c1d13","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.278Z","job_size_bytes":3,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-de8824b13815475a5ecc65a4: done: 0.029867 sec","job_status":"done","queue_duration_s":1.834498,"scheduling_latency_s":1.834589,"redis_calls":10,"redis_duration_s":0.002283,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000754,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000324,"duration_s":0.029867,"completed_at":"2026-11-07T11:04:05.142Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.524182] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"20d81d643d779114e60ff893","created_at":"2026-11-07T11:04:03.296Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73ea2f41c7fce8def6ca48f1f64bdb80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.303Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-20d81d643d779114e60ff893: start","job_status":"start","queue_duration_s":1.841165,"scheduling_latency_s":1.841195} vm-test-run-nixbot-gitlab> gitlab # [ 330.798944] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"20d81d643d779114e60ff893","created_at":"2026-11-07T11:04:03.296Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73ea2f41c7fce8def6ca48f1f64bdb80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.303Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-20d81d643d779114e60ff893: done: 0.377894 sec","job_status":"done","queue_duration_s":1.841165,"scheduling_latency_s":1.841195,"redis_calls":11,"redis_duration_s":0.004058,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.32092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008404,"exclusive_lock_hold_duration_s":0.34725404,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.377894,"completed_at":"2026-11-07T11:04:05.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069664,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 330.873028] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"138c5130b7da1c5db24ec3c1","created_at":"2026-11-07T11:04:03.314Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.329Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-138c5130b7da1c5db24ec3c1: start","job_status":"start","queue_duration_s":2.194738,"scheduling_latency_s":2.19477} vm-test-run-nixbot-gitlab> gitlab # [ 331.070931] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"138c5130b7da1c5db24ec3c1","created_at":"2026-11-07T11:04:03.314Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.329Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-138c5130b7da1c5db24ec3c1: done: 0.269778 sec","job_status":"done","queue_duration_s":2.194738,"scheduling_latency_s":2.19477,"redis_calls":60,"redis_duration_s":0.093507,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.002953,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.086451,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004103,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.269778,"completed_at":"2026-11-07T11:04:05.793Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.137387] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9cd7176335f39e9d5b86fc52","created_at":"2026-11-07T11:04:03.338Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53894966de3e181e2367eea2eef72083","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.348Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9cd7176335f39e9d5b86fc52: start","job_status":"start","queue_duration_s":2.453445,"scheduling_latency_s":2.45353} vm-test-run-nixbot-gitlab> gitlab # [ 331.158280] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.826Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 331.162588] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9cd7176335f39e9d5b86fc52","created_at":"2026-11-07T11:04:03.338Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53894966de3e181e2367eea2eef72083","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.348Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9cd7176335f39e9d5b86fc52: done: 0.055487 sec","job_status":"done","queue_duration_s":2.453445,"scheduling_latency_s":2.45353,"redis_calls":7,"redis_duration_s":0.005595,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00375,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055487,"completed_at":"2026-11-07T11:04:05.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.206601] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33ba672b5fd0eef703148ebb","created_at":"2026-11-07T11:04:03.368Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d84680e005646599db414ffdf70735d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.423Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33ba672b5fd0eef703148ebb: start","job_status":"start","queue_duration_s":2.437954,"scheduling_latency_s":2.438002} vm-test-run-nixbot-gitlab> gitlab # [ 331.221924] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33ba672b5fd0eef703148ebb","created_at":"2026-11-07T11:04:03.368Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d84680e005646599db414ffdf70735d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.423Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33ba672b5fd0eef703148ebb: done: 0.049875 sec","job_status":"done","queue_duration_s":2.437954,"scheduling_latency_s":2.438002,"redis_calls":7,"redis_duration_s":0.002702,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001875,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049875,"completed_at":"2026-11-07T11:04:05.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.264926] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"47f8b7d7a19f0428e7ac5e24","created_at":"2026-11-07T11:04:03.442Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7a0cade2c9bb5208463ba403d57aa04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.450Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-47f8b7d7a19f0428e7ac5e24: start","job_status":"start","queue_duration_s":2.46692,"scheduling_latency_s":2.466967} vm-test-run-nixbot-gitlab> gitlab # [ 331.279982] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.943Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 331.284084] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"47f8b7d7a19f0428e7ac5e24","created_at":"2026-11-07T11:04:03.442Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7a0cade2c9bb5208463ba403d57aa04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.450Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-47f8b7d7a19f0428e7ac5e24: done: 0.048917 sec","job_status":"done","queue_duration_s":2.46692,"scheduling_latency_s":2.466967,"redis_calls":7,"redis_duration_s":0.002671,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048917,"completed_at":"2026-11-07T11:04:05.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.326323] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:05.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d1abe088052103037ac3d5ee","created_at":"2026-11-07T11:04:03.462Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1dceaa1e216ba5c185adcf68e4372f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.468Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d1abe088052103037ac3d5ee: start","job_status":"start","queue_duration_s":2.500272,"scheduling_latency_s":2.500318} vm-test-run-nixbot-gitlab> gitlab # [ 331.341038] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d1abe088052103037ac3d5ee","created_at":"2026-11-07T11:04:03.462Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1dceaa1e216ba5c185adcf68e4372f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.468Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d1abe088052103037ac3d5ee: done: 0.0818 sec","job_status":"done","queue_duration_s":2.500272,"scheduling_latency_s":2.500318,"redis_calls":7,"redis_duration_s":0.002477,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0818,"completed_at":"2026-11-07T11:04:06.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.386312] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"62c6549d36a56fd8f90ede32","created_at":"2026-11-07T11:04:03.610Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d23fbdbf71bd1b9f3623e3f3714d33e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.618Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-62c6549d36a56fd8f90ede32: start","job_status":"start","queue_duration_s":2.43518,"scheduling_latency_s":2.435219} vm-test-run-nixbot-gitlab> gitlab # [ 331.459977] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"62c6549d36a56fd8f90ede32","created_at":"2026-11-07T11:04:03.610Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d23fbdbf71bd1b9f3623e3f3714d33e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.618Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-62c6549d36a56fd8f90ede32: done: 0.129876 sec","job_status":"done","queue_duration_s":2.43518,"scheduling_latency_s":2.435219,"redis_calls":9,"redis_duration_s":0.0034270000000000004,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001688,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00115548,"exclusive_lock_hold_duration_s":0.00728564,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.129876,"completed_at":"2026-11-07T11:04:06.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.520468] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3aefdd77c0373091bb33c2e","created_at":"2026-11-07T11:04:03.657Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2bcf4706fa9cf1cc1f1571f6c7a48ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.663Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3aefdd77c0373091bb33c2e: start","job_status":"start","queue_duration_s":2.521608,"scheduling_latency_s":2.521646} vm-test-run-nixbot-gitlab> gitlab # [ 331.561819] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3aefdd77c0373091bb33c2e","created_at":"2026-11-07T11:04:03.657Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2bcf4706fa9cf1cc1f1571f6c7a48ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.663Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3aefdd77c0373091bb33c2e: done: 0.099573 sec","job_status":"done","queue_duration_s":2.521608,"scheduling_latency_s":2.521646,"redis_calls":7,"redis_duration_s":0.00251,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099573,"completed_at":"2026-11-07T11:04:06.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.620106] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"65b7fb2955189bc04a8c0c48","created_at":"2026-11-07T11:04:03.682Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74cbeed5d215d18de2fd5afe455b943c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.688Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-65b7fb2955189bc04a8c0c48: start","job_status":"start","queue_duration_s":2.602384,"scheduling_latency_s":2.602438} vm-test-run-nixbot-gitlab> gitlab # [ 331.891908] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"65b7fb2955189bc04a8c0c48","created_at":"2026-11-07T11:04:03.682Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74cbeed5d215d18de2fd5afe455b943c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.688Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-65b7fb2955189bc04a8c0c48: done: 0.322423 sec","job_status":"done","queue_duration_s":2.602384,"scheduling_latency_s":2.602438,"redis_calls":9,"redis_duration_s":0.013688,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008302,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00139122,"exclusive_lock_hold_duration_s":0.23684212,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.322423,"completed_at":"2026-11-07T11:04:06.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.225366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 331.958811] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1d03c540133b338df2df69ac","created_at":"2026-11-07T11:04:03.717Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.729Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1d03c540133b338df2df69ac: start","job_status":"start","queue_duration_s":2.894289,"scheduling_latency_s":2.894338} vm-test-run-nixbot-gitlab> gitlab # [ 332.038504] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1d03c540133b338df2df69ac","created_at":"2026-11-07T11:04:03.717Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:04:03.729Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1d03c540133b338df2df69ac: done: 0.138129 sec","job_status":"done","queue_duration_s":2.894289,"scheduling_latency_s":2.894338,"redis_calls":14,"redis_duration_s":0.023002,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.001028,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.019834,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00214,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138129,"completed_at":"2026-11-07T11:04:06.761Z","load_balancing_strategy":"primary","db_duration_s":0.011056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.103886] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d0f3f71097dbe61babfc425","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d0f3f71097dbe61babfc425: start","job_status":"start","queue_duration_s":1.021464,"scheduling_latency_s":1.021515} vm-test-run-nixbot-gitlab> gitlab # [ 332.142325] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d0f3f71097dbe61babfc425","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d0f3f71097dbe61babfc425: done: 0.091675 sec","job_status":"done","queue_duration_s":1.021464,"scheduling_latency_s":1.021515,"redis_calls":13,"redis_duration_s":0.039213,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.026631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.091675,"completed_at":"2026-11-07T11:04:06.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.195906] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a273651e12819470a242f70c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a273651e12819470a242f70c: start","job_status":"start","queue_duration_s":1.12465,"scheduling_latency_s":1.124702} vm-test-run-nixbot-gitlab> gitlab # [ 332.257431] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a273651e12819470a242f70c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a273651e12819470a242f70c: done: 0.102614 sec","job_status":"done","queue_duration_s":1.12465,"scheduling_latency_s":1.124702,"redis_calls":13,"redis_duration_s":0.032442,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.021289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.102614,"completed_at":"2026-11-07T11:04:06.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.312509] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:06.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a64b59e66b070fc3dd4d0801","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a64b59e66b070fc3dd4d0801: start","job_status":"start","queue_duration_s":1.240841,"scheduling_latency_s":1.240893} vm-test-run-nixbot-gitlab> gitlab # [ 332.412100] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a64b59e66b070fc3dd4d0801","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a64b59e66b070fc3dd4d0801: done: 0.139775 sec","job_status":"done","queue_duration_s":1.240841,"scheduling_latency_s":1.240893,"redis_calls":13,"redis_duration_s":0.038858000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.028121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.139775,"completed_at":"2026-11-07T11:04:07.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.463816] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b27cabf6dea84a162aee09","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b27cabf6dea84a162aee09: start","job_status":"start","queue_duration_s":1.387287,"scheduling_latency_s":1.387335} vm-test-run-nixbot-gitlab> gitlab # [ 332.540787] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24b27cabf6dea84a162aee09","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24b27cabf6dea84a162aee09: done: 0.124353 sec","job_status":"done","queue_duration_s":1.387287,"scheduling_latency_s":1.387335,"redis_calls":13,"redis_duration_s":0.058202000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.041937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.124353,"completed_at":"2026-11-07T11:04:07.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.601084] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1b00d5b832b2489f05ffdfa","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1b00d5b832b2489f05ffdfa: start","job_status":"start","queue_duration_s":1.523942,"scheduling_latency_s":1.523994} vm-test-run-nixbot-gitlab> gitlab # [ 332.652060] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1b00d5b832b2489f05ffdfa","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1b00d5b832b2489f05ffdfa: done: 0.09875 sec","job_status":"done","queue_duration_s":1.523942,"scheduling_latency_s":1.523994,"redis_calls":13,"redis_duration_s":0.040644,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.01304,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.027604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.09875,"completed_at":"2026-11-07T11:04:07.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.715535] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a87f3939d265a501db30015c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a87f3939d265a501db30015c: start","job_status":"start","queue_duration_s":1.638393,"scheduling_latency_s":1.638449} vm-test-run-nixbot-gitlab> gitlab # [ 332.874407] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a87f3939d265a501db30015c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a87f3939d265a501db30015c: done: 0.207054 sec","job_status":"done","queue_duration_s":1.638393,"scheduling_latency_s":1.638449,"redis_calls":13,"redis_duration_s":0.07103100000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.048765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.022266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.207054,"completed_at":"2026-11-07T11:04:07.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 332.937976] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"499eb4dfdff8cb9f81918922","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-499eb4dfdff8cb9f81918922: start","job_status":"start","queue_duration_s":1.855206,"scheduling_latency_s":1.855255} vm-test-run-nixbot-gitlab> gitlab # [ 332.977149] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"499eb4dfdff8cb9f81918922","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-499eb4dfdff8cb9f81918922: done: 0.09264 sec","job_status":"done","queue_duration_s":1.855206,"scheduling_latency_s":1.855255,"redis_calls":13,"redis_duration_s":0.034694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010357,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.024337,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.09264,"completed_at":"2026-11-07T11:04:07.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.033872] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c1e5445a958422c88e21774","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c1e5445a958422c88e21774: start","job_status":"start","queue_duration_s":1.959317,"scheduling_latency_s":1.959369} vm-test-run-nixbot-gitlab> gitlab # [ 333.085889] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c1e5445a958422c88e21774","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c1e5445a958422c88e21774: done: 0.097598 sec","job_status":"done","queue_duration_s":1.959317,"scheduling_latency_s":1.959369,"redis_calls":13,"redis_duration_s":0.037753999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010175,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.027579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.097598,"completed_at":"2026-11-07T11:04:07.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.146131] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64c5847903900045905c189d","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64c5847903900045905c189d: start","job_status":"start","queue_duration_s":2.070662,"scheduling_latency_s":2.070714} vm-test-run-nixbot-gitlab> gitlab # [ 333.193855] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64c5847903900045905c189d","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64c5847903900045905c189d: done: 0.094326 sec","job_status":"done","queue_duration_s":2.070662,"scheduling_latency_s":2.070714,"redis_calls":13,"redis_duration_s":0.037376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.026359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.094326,"completed_at":"2026-11-07T11:04:07.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.253056] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:07.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da2863335fc1d3a1aeab6874","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da2863335fc1d3a1aeab6874: start","job_status":"start","queue_duration_s":2.173234,"scheduling_latency_s":2.173281} vm-test-run-nixbot-gitlab> gitlab # [ 333.290686] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da2863335fc1d3a1aeab6874","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da2863335fc1d3a1aeab6874: done: 0.087068 sec","job_status":"done","queue_duration_s":2.173234,"scheduling_latency_s":2.173281,"redis_calls":13,"redis_duration_s":0.009895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.087068,"completed_at":"2026-11-07T11:04:08.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.349836] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2932752cd08fb5972100d36","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2932752cd08fb5972100d36: start","job_status":"start","queue_duration_s":2.265582,"scheduling_latency_s":2.265632} vm-test-run-nixbot-gitlab> gitlab # [ 333.409430] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2932752cd08fb5972100d36","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2932752cd08fb5972100d36: done: 0.114898 sec","job_status":"done","queue_duration_s":2.265582,"scheduling_latency_s":2.265632,"redis_calls":13,"redis_duration_s":0.026646999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018601,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.114898,"completed_at":"2026-11-07T11:04:08.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.476462] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95940fec97809989edac2465","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95940fec97809989edac2465: start","job_status":"start","queue_duration_s":2.385143,"scheduling_latency_s":2.385194} vm-test-run-nixbot-gitlab> gitlab # [ 333.494064] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95940fec97809989edac2465","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95940fec97809989edac2465: done: 0.073218 sec","job_status":"done","queue_duration_s":2.385143,"scheduling_latency_s":2.385194,"redis_calls":13,"redis_duration_s":0.007967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006393,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.073218,"completed_at":"2026-11-07T11:04:08.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.541661] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"952ac61c52c4b3ad61d25a3f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-952ac61c52c4b3ad61d25a3f: start","job_status":"start","queue_duration_s":2.46608,"scheduling_latency_s":2.46613} vm-test-run-nixbot-gitlab> gitlab # [ 333.585898] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"952ac61c52c4b3ad61d25a3f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-952ac61c52c4b3ad61d25a3f: done: 0.090956 sec","job_status":"done","queue_duration_s":2.46608,"scheduling_latency_s":2.46613,"redis_calls":13,"redis_duration_s":0.009094,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006816,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.090956,"completed_at":"2026-11-07T11:04:08.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.634046] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb210d3d2f621b27d8691bce","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb210d3d2f621b27d8691bce: start","job_status":"start","queue_duration_s":2.560909,"scheduling_latency_s":2.560952} vm-test-run-nixbot-gitlab> gitlab # [ 333.655290] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb210d3d2f621b27d8691bce","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb210d3d2f621b27d8691bce: done: 0.06544 sec","job_status":"done","queue_duration_s":2.560909,"scheduling_latency_s":2.560952,"redis_calls":13,"redis_duration_s":0.003966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.06544,"completed_at":"2026-11-07T11:04:08.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.698765] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c84e4a9cf938cc41f205d26","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c84e4a9cf938cc41f205d26: start","job_status":"start","queue_duration_s":2.630195,"scheduling_latency_s":2.63024} vm-test-run-nixbot-gitlab> gitlab # [ 333.808832] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c84e4a9cf938cc41f205d26","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c84e4a9cf938cc41f205d26: done: 0.148099 sec","job_status":"done","queue_duration_s":2.630195,"scheduling_latency_s":2.63024,"redis_calls":13,"redis_duration_s":0.010291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.148099,"completed_at":"2026-11-07T11:04:08.530Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.871123] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911437156db17ff29bd51683","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911437156db17ff29bd51683: start","job_status":"start","queue_duration_s":2.78389,"scheduling_latency_s":2.783989} vm-test-run-nixbot-gitlab> gitlab # [ 333.897328] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911437156db17ff29bd51683","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911437156db17ff29bd51683: done: 0.083259 sec","job_status":"done","queue_duration_s":2.78389,"scheduling_latency_s":2.783989,"redis_calls":13,"redis_duration_s":0.010772,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.083259,"completed_at":"2026-11-07T11:04:08.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 333.950263] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"327317499e2edf20ab7889b5","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-327317499e2edf20ab7889b5: start","job_status":"start","queue_duration_s":2.872661,"scheduling_latency_s":2.872719} vm-test-run-nixbot-gitlab> gitlab # [ 333.999115] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"327317499e2edf20ab7889b5","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-327317499e2edf20ab7889b5: done: 0.097591 sec","job_status":"done","queue_duration_s":2.872661,"scheduling_latency_s":2.872719,"redis_calls":13,"redis_duration_s":0.005062,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.097591,"completed_at":"2026-11-07T11:04:08.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 334.056288] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df490687a18e0878ba6abfad","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df490687a18e0878ba6abfad: start","job_status":"start","queue_duration_s":2.97311,"scheduling_latency_s":2.97316} vm-test-run-nixbot-gitlab> gitlab # [ 334.080433] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df490687a18e0878ba6abfad","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df490687a18e0878ba6abfad: done: 0.077965 sec","job_status":"done","queue_duration_s":2.97311,"scheduling_latency_s":2.97316,"redis_calls":13,"redis_duration_s":0.004340999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.077965,"completed_at":"2026-11-07T11:04:08.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 334.129467] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26f56ced2b3d44350dfaf6e6","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26f56ced2b3d44350dfaf6e6: start","job_status":"start","queue_duration_s":3.057061,"scheduling_latency_s":3.057116} vm-test-run-nixbot-gitlab> gitlab # [ 334.257331] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26f56ced2b3d44350dfaf6e6","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26f56ced2b3d44350dfaf6e6: done: 0.170483 sec","job_status":"done","queue_duration_s":3.057061,"scheduling_latency_s":3.057116,"redis_calls":13,"redis_duration_s":0.02085,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.170483,"completed_at":"2026-11-07T11:04:08.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 334.318844] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:08.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c3913853ba51ddbb1f5324c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c3913853ba51ddbb1f5324c: start","job_status":"start","queue_duration_s":3.240922,"scheduling_latency_s":3.240975} vm-test-run-nixbot-gitlab> gitlab # [ 334.418378] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c3913853ba51ddbb1f5324c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c3913853ba51ddbb1f5324c: done: 0.147407 sec","job_status":"done","queue_duration_s":3.240922,"scheduling_latency_s":3.240975,"redis_calls":13,"redis_duration_s":0.041873999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.016138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.025736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.147407,"completed_at":"2026-11-07T11:04:09.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 334.469201] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90387ed3a3e927789f146fcd","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90387ed3a3e927789f146fcd: start","job_status":"start","queue_duration_s":3.400388,"scheduling_latency_s":3.40044} vm-test-run-nixbot-gitlab> gitlab # [ 334.775845] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90387ed3a3e927789f146fcd","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90387ed3a3e927789f146fcd: done: 0.345078 sec","job_status":"done","queue_duration_s":3.400388,"scheduling_latency_s":3.40044,"redis_calls":13,"redis_duration_s":0.039821999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.028338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.345078,"completed_at":"2026-11-07T11:04:09.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 334.838767] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e4f159e109fdc02bd3be36f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e4f159e109fdc02bd3be36f: start","job_status":"start","queue_duration_s":3.797587,"scheduling_latency_s":3.797776} vm-test-run-nixbot-gitlab> gitlab # [ 334.995068] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e4f159e109fdc02bd3be36f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e4f159e109fdc02bd3be36f: done: 0.168464 sec","job_status":"done","queue_duration_s":3.797587,"scheduling_latency_s":3.797776,"redis_calls":13,"redis_duration_s":0.033366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.168464,"completed_at":"2026-11-07T11:04:09.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.065079] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc88c7bd6516704cee4b0a74","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc88c7bd6516704cee4b0a74: start","job_status":"start","queue_duration_s":4.036135,"scheduling_latency_s":4.036191} vm-test-run-nixbot-gitlab> gitlab # [ 335.184178] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:09.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc88c7bd6516704cee4b0a74","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc88c7bd6516704cee4b0a74: done: 0.117849 sec","job_status":"done","queue_duration_s":4.036135,"scheduling_latency_s":4.036191,"redis_calls":13,"redis_duration_s":0.027077999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.020255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.117849,"completed_at":"2026-11-07T11:04:09.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.294661] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04aaa8b688da80b07fbcd884","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04aaa8b688da80b07fbcd884: start","job_status":"start","queue_duration_s":4.265168,"scheduling_latency_s":4.26522} vm-test-run-nixbot-gitlab> gitlab # [ 335.402124] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04aaa8b688da80b07fbcd884","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04aaa8b688da80b07fbcd884: done: 0.107617 sec","job_status":"done","queue_duration_s":4.265168,"scheduling_latency_s":4.26522,"redis_calls":13,"redis_duration_s":0.028089000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010301,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.017788,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.107617,"completed_at":"2026-11-07T11:04:10.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.459317] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cf27c5aa183a7d36b9df09b","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cf27c5aa183a7d36b9df09b: start","job_status":"start","queue_duration_s":4.389708,"scheduling_latency_s":4.389761} vm-test-run-nixbot-gitlab> gitlab # [ 335.520175] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cf27c5aa183a7d36b9df09b","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cf27c5aa183a7d36b9df09b: done: 0.098282 sec","job_status":"done","queue_duration_s":4.389708,"scheduling_latency_s":4.389761,"redis_calls":13,"redis_duration_s":0.033462000000000006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009159,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.024303,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.098282,"completed_at":"2026-11-07T11:04:10.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.574494] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ac969bbb8bc7279796d3ebf","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ac969bbb8bc7279796d3ebf: start","job_status":"start","queue_duration_s":4.508153,"scheduling_latency_s":4.508208} vm-test-run-nixbot-gitlab> gitlab # [ 335.621540] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ac969bbb8bc7279796d3ebf","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ac969bbb8bc7279796d3ebf: done: 0.084114 sec","job_status":"done","queue_duration_s":4.508153,"scheduling_latency_s":4.508208,"redis_calls":13,"redis_duration_s":0.009647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005749,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.084114,"completed_at":"2026-11-07T11:04:10.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.674930] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23366e80b07e43b0f8080585","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23366e80b07e43b0f8080585: start","job_status":"start","queue_duration_s":4.598013,"scheduling_latency_s":4.59806} vm-test-run-nixbot-gitlab> gitlab # [ 335.742466] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23366e80b07e43b0f8080585","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23366e80b07e43b0f8080585: done: 0.113769 sec","job_status":"done","queue_duration_s":4.598013,"scheduling_latency_s":4.59806,"redis_calls":13,"redis_duration_s":0.047673999999999994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.041953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.113769,"completed_at":"2026-11-07T11:04:10.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.802603] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c4517badd1ca20b678f464a","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c4517badd1ca20b678f464a: start","job_status":"start","queue_duration_s":4.769273,"scheduling_latency_s":4.769329} vm-test-run-nixbot-gitlab> gitlab # [ 335.919854] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c4517badd1ca20b678f464a","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c4517badd1ca20b678f464a: done: 0.118461 sec","job_status":"done","queue_duration_s":4.769273,"scheduling_latency_s":4.769329,"redis_calls":13,"redis_duration_s":0.027826000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007346,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.02048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.118461,"completed_at":"2026-11-07T11:04:10.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 335.973934] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"733cd89592c8f3e77509e35f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-733cd89592c8f3e77509e35f: start","job_status":"start","queue_duration_s":4.90169,"scheduling_latency_s":4.901742} vm-test-run-nixbot-gitlab> gitlab # [ 336.275627] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:10.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"733cd89592c8f3e77509e35f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-733cd89592c8f3e77509e35f: done: 0.339508 sec","job_status":"done","queue_duration_s":4.90169,"scheduling_latency_s":4.901742,"redis_calls":13,"redis_duration_s":0.132441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.1214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.339508,"completed_at":"2026-11-07T11:04:10.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.321135] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844efd6a2c31939324af4c90","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844efd6a2c31939324af4c90: start","job_status":"start","queue_duration_s":5.256585,"scheduling_latency_s":5.256633} vm-test-run-nixbot-gitlab> gitlab # [ 336.404164] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844efd6a2c31939324af4c90","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844efd6a2c31939324af4c90: done: 0.116938 sec","job_status":"done","queue_duration_s":5.256585,"scheduling_latency_s":5.256633,"redis_calls":13,"redis_duration_s":0.027599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.017641,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.116938,"completed_at":"2026-11-07T11:04:11.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.465532] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930747cf307d76c5f24adeb4","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930747cf307d76c5f24adeb4: start","job_status":"start","queue_duration_s":5.378591,"scheduling_latency_s":5.37864} vm-test-run-nixbot-gitlab> gitlab # [ 336.498450] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930747cf307d76c5f24adeb4","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930747cf307d76c5f24adeb4: done: 0.090816 sec","job_status":"done","queue_duration_s":5.378591,"scheduling_latency_s":5.37864,"redis_calls":13,"redis_duration_s":0.013255,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011725,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.090816,"completed_at":"2026-11-07T11:04:11.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.553720] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"182b151cf36620176fea8386","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-182b151cf36620176fea8386: start","job_status":"start","queue_duration_s":5.474536,"scheduling_latency_s":5.474582} vm-test-run-nixbot-gitlab> gitlab # [ 336.614684] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"182b151cf36620176fea8386","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-182b151cf36620176fea8386: done: 0.109336 sec","job_status":"done","queue_duration_s":5.474536,"scheduling_latency_s":5.474582,"redis_calls":13,"redis_duration_s":0.005882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.109336,"completed_at":"2026-11-07T11:04:11.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.679006] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcd9d8afb73bc9ca2e04bd0c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcd9d8afb73bc9ca2e04bd0c: start","job_status":"start","queue_duration_s":5.592113,"scheduling_latency_s":5.592255} vm-test-run-nixbot-gitlab> gitlab # [ 336.697097] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcd9d8afb73bc9ca2e04bd0c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcd9d8afb73bc9ca2e04bd0c: done: 0.035939 sec","job_status":"done","queue_duration_s":5.592113,"scheduling_latency_s":5.592255,"redis_calls":13,"redis_duration_s":0.0032710000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035939,"completed_at":"2026-11-07T11:04:11.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.740980] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a63486fc03e74c2781f2034","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a63486fc03e74c2781f2034: start","job_status":"start","queue_duration_s":5.630102,"scheduling_latency_s":5.63014} vm-test-run-nixbot-gitlab> gitlab # [ 336.753750] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a63486fc03e74c2781f2034","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a63486fc03e74c2781f2034: done: 0.027753 sec","job_status":"done","queue_duration_s":5.630102,"scheduling_latency_s":5.63014,"redis_calls":13,"redis_duration_s":0.0030340000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027753,"completed_at":"2026-11-07T11:04:11.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.795322] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0051f357b86058d09a042421","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0051f357b86058d09a042421: start","job_status":"start","queue_duration_s":5.659899,"scheduling_latency_s":5.659966} vm-test-run-nixbot-gitlab> gitlab # [ 336.808448] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0051f357b86058d09a042421","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0051f357b86058d09a042421: done: 0.038161 sec","job_status":"done","queue_duration_s":5.659899,"scheduling_latency_s":5.659966,"redis_calls":13,"redis_duration_s":0.004008,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002542,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038161,"completed_at":"2026-11-07T11:04:11.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.849168] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e22b2097dd74b20e077ad00","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e22b2097dd74b20e077ad00: start","job_status":"start","queue_duration_s":5.700052,"scheduling_latency_s":5.700088} vm-test-run-nixbot-gitlab> gitlab # [ 336.861939] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e22b2097dd74b20e077ad00","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e22b2097dd74b20e077ad00: done: 0.025997 sec","job_status":"done","queue_duration_s":5.700052,"scheduling_latency_s":5.700088,"redis_calls":13,"redis_duration_s":0.0027670000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001765,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025997,"completed_at":"2026-11-07T11:04:11.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.902394] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98047b93542fb9ab1ba51df1","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98047b93542fb9ab1ba51df1: start","job_status":"start","queue_duration_s":5.728109,"scheduling_latency_s":5.728146} vm-test-run-nixbot-gitlab> gitlab # [ 336.915316] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98047b93542fb9ab1ba51df1","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98047b93542fb9ab1ba51df1: done: 0.026661 sec","job_status":"done","queue_duration_s":5.728109,"scheduling_latency_s":5.728146,"redis_calls":13,"redis_duration_s":0.003039,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026661,"completed_at":"2026-11-07T11:04:11.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 336.956179] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7da559e0ddf9f55025ff972a","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7da559e0ddf9f55025ff972a: start","job_status":"start","queue_duration_s":5.756706,"scheduling_latency_s":5.75674} vm-test-run-nixbot-gitlab> gitlab # [ 336.968684] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7da559e0ddf9f55025ff972a","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7da559e0ddf9f55025ff972a: done: 0.025306 sec","job_status":"done","queue_duration_s":5.756706,"scheduling_latency_s":5.75674,"redis_calls":13,"redis_duration_s":0.00332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002355,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025306,"completed_at":"2026-11-07T11:04:11.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.010267] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0561ce06fa03622034a1f5f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0561ce06fa03622034a1f5f: start","job_status":"start","queue_duration_s":5.784012,"scheduling_latency_s":5.784048} vm-test-run-nixbot-gitlab> gitlab # [ 337.023063] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0561ce06fa03622034a1f5f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0561ce06fa03622034a1f5f: done: 0.024212 sec","job_status":"done","queue_duration_s":5.784012,"scheduling_latency_s":5.784048,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024212,"completed_at":"2026-11-07T11:04:11.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.063681] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e961efe9609e3c29c196471d","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e961efe9609e3c29c196471d: start","job_status":"start","queue_duration_s":5.810182,"scheduling_latency_s":5.810216} vm-test-run-nixbot-gitlab> gitlab # [ 337.076671] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e961efe9609e3c29c196471d","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e961efe9609e3c29c196471d: done: 0.024249 sec","job_status":"done","queue_duration_s":5.810182,"scheduling_latency_s":5.810216,"redis_calls":13,"redis_duration_s":0.002686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001667,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024249,"completed_at":"2026-11-07T11:04:11.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.117017] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ad8438e574a02d46d81411","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ad8438e574a02d46d81411: start","job_status":"start","queue_duration_s":5.836655,"scheduling_latency_s":5.836689} vm-test-run-nixbot-gitlab> gitlab # [ 337.129683] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ad8438e574a02d46d81411","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ad8438e574a02d46d81411: done: 0.023668 sec","job_status":"done","queue_duration_s":5.836655,"scheduling_latency_s":5.836689,"redis_calls":13,"redis_duration_s":0.002328,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023668,"completed_at":"2026-11-07T11:04:11.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.170075] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"553fa2069e3e12d5a0a8bf7e","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-553fa2069e3e12d5a0a8bf7e: start","job_status":"start","queue_duration_s":5.862195,"scheduling_latency_s":5.862228} vm-test-run-nixbot-gitlab> gitlab # [ 337.182864] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"553fa2069e3e12d5a0a8bf7e","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-553fa2069e3e12d5a0a8bf7e: done: 0.024939 sec","job_status":"done","queue_duration_s":5.862195,"scheduling_latency_s":5.862228,"redis_calls":13,"redis_duration_s":0.002977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001677,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024939,"completed_at":"2026-11-07T11:04:11.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.224857] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831995fa6576e043fbe62a3f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831995fa6576e043fbe62a3f: start","job_status":"start","queue_duration_s":5.88889,"scheduling_latency_s":5.888921} vm-test-run-nixbot-gitlab> gitlab # [ 337.239431] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831995fa6576e043fbe62a3f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831995fa6576e043fbe62a3f: done: 0.018797 sec","job_status":"done","queue_duration_s":5.88889,"scheduling_latency_s":5.888921,"redis_calls":13,"redis_duration_s":0.0018349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018797,"completed_at":"2026-11-07T11:04:11.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.282062] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"807f4b8d002129fcdf5504a2","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-807f4b8d002129fcdf5504a2: start","job_status":"start","queue_duration_s":5.909131,"scheduling_latency_s":5.909157} vm-test-run-nixbot-gitlab> gitlab # [ 337.294849] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"807f4b8d002129fcdf5504a2","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-807f4b8d002129fcdf5504a2: done: 0.017997 sec","job_status":"done","queue_duration_s":5.909131,"scheduling_latency_s":5.909157,"redis_calls":13,"redis_duration_s":0.001759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017997,"completed_at":"2026-11-07T11:04:11.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.335853] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c551c39b641da267f51aaa8","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c551c39b641da267f51aaa8: start","job_status":"start","queue_duration_s":5.928671,"scheduling_latency_s":5.928697} vm-test-run-nixbot-gitlab> gitlab # [ 337.348926] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c551c39b641da267f51aaa8","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c551c39b641da267f51aaa8: done: 0.018576 sec","job_status":"done","queue_duration_s":5.928671,"scheduling_latency_s":5.928697,"redis_calls":13,"redis_duration_s":0.001944,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018576,"completed_at":"2026-11-07T11:04:11.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.389173] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93bd3b670d599df42cc9de03","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93bd3b670d599df42cc9de03: start","job_status":"start","queue_duration_s":5.948882,"scheduling_latency_s":5.948909} vm-test-run-nixbot-gitlab> gitlab # [ 337.401910] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93bd3b670d599df42cc9de03","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93bd3b670d599df42cc9de03: done: 0.018465 sec","job_status":"done","queue_duration_s":5.948882,"scheduling_latency_s":5.948909,"redis_calls":13,"redis_duration_s":0.0019390000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018465,"completed_at":"2026-11-07T11:04:11.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.443507] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bb42efb1750e3364a3330c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bb42efb1750e3364a3330c: start","job_status":"start","queue_duration_s":5.968995,"scheduling_latency_s":5.969023} vm-test-run-nixbot-gitlab> gitlab # [ 337.456413] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bb42efb1750e3364a3330c","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bb42efb1750e3364a3330c: done: 0.020051 sec","job_status":"done","queue_duration_s":5.968995,"scheduling_latency_s":5.969023,"redis_calls":13,"redis_duration_s":0.002154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020051,"completed_at":"2026-11-07T11:04:11.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.496891] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ccc97b4baaa864c417f27f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ccc97b4baaa864c417f27f: start","job_status":"start","queue_duration_s":5.990666,"scheduling_latency_s":5.990693} vm-test-run-nixbot-gitlab> gitlab # [ 337.509598] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ccc97b4baaa864c417f27f","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ccc97b4baaa864c417f27f: done: 0.01896 sec","job_status":"done","queue_duration_s":5.990666,"scheduling_latency_s":5.990693,"redis_calls":13,"redis_duration_s":0.001957,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01896,"completed_at":"2026-11-07T11:04:11.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.549859] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e072d57fbec6e3e421568f4","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e072d57fbec6e3e421568f4: start","job_status":"start","queue_duration_s":6.011203,"scheduling_latency_s":6.011228} vm-test-run-nixbot-gitlab> gitlab # [ 337.562613] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e072d57fbec6e3e421568f4","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e072d57fbec6e3e421568f4: done: 0.0187 sec","job_status":"done","queue_duration_s":6.011203,"scheduling_latency_s":6.011228,"redis_calls":13,"redis_duration_s":0.0019119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0187,"completed_at":"2026-11-07T11:04:11.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.603209] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67c1d3bb2b77b13fbd019454","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67c1d3bb2b77b13fbd019454: start","job_status":"start","queue_duration_s":6.031474,"scheduling_latency_s":6.031501} vm-test-run-nixbot-gitlab> gitlab # [ 337.617174] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67c1d3bb2b77b13fbd019454","created_at":"2026-11-07T11:04:05.535Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-6a5ae3ef1709499a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3ea13930f6dc53e52bcb957b1e681044","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:05.752Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67c1d3bb2b77b13fbd019454: done: 0.019516 sec","job_status":"done","queue_duration_s":6.031474,"scheduling_latency_s":6.031501,"redis_calls":13,"redis_duration_s":0.002134,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019516,"completed_at":"2026-11-07T11:04:11.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.657528] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.804Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"08e10a9d257c64029a4b0da1","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-08e10a9d257c64029a4b0da1: start","job_status":"start","queue_duration_s":5.074711,"scheduling_latency_s":5.074737} vm-test-run-nixbot-gitlab> gitlab # [ 337.670219] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.819Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 337.671931] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.832Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"08e10a9d257c64029a4b0da1","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-08e10a9d257c64029a4b0da1: done: 0.027309 sec","job_status":"done","queue_duration_s":5.074711,"scheduling_latency_s":5.074737,"redis_calls":10,"redis_duration_s":0.002242,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027309,"completed_at":"2026-11-07T11:04:11.832Z","load_balancing_strategy":"primary","db_duration_s":0.002712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.711914] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.833Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd9e9558cc08e8826e6f861f","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd9e9558cc08e8826e6f861f: start","job_status":"start","queue_duration_s":5.103603,"scheduling_latency_s":5.103628} vm-test-run-nixbot-gitlab> gitlab # [ 337.724832] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.847Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 337.726549] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.859Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd9e9558cc08e8826e6f861f","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd9e9558cc08e8826e6f861f: done: 0.025565 sec","job_status":"done","queue_duration_s":5.103603,"scheduling_latency_s":5.103628,"redis_calls":10,"redis_duration_s":0.002116,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025565,"completed_at":"2026-11-07T11:04:11.859Z","load_balancing_strategy":"primary","db_duration_s":0.002173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.766241] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.861Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea7a5130d561df1f39184c9b","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea7a5130d561df1f39184c9b: start","job_status":"start","queue_duration_s":5.130854,"scheduling_latency_s":5.130881} vm-test-run-nixbot-gitlab> gitlab # [ 337.778896] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.874Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 337.780674] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.886Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea7a5130d561df1f39184c9b","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea7a5130d561df1f39184c9b: done: 0.025093 sec","job_status":"done","queue_duration_s":5.130854,"scheduling_latency_s":5.130881,"redis_calls":10,"redis_duration_s":0.0019060000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025093,"completed_at":"2026-11-07T11:04:11.885Z","load_balancing_strategy":"primary","db_duration_s":0.002181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 337.821602] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"862220bf343a6df9f07f5619","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-862220bf343a6df9f07f5619: start","job_status":"start","queue_duration_s":5.157619,"scheduling_latency_s":5.157648} vm-test-run-nixbot-gitlab> gitlab # [ 337.834301] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.902Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 337.836005] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:04:11.913Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"862220bf343a6df9f07f5619","created_at":"2026-11-07T11:04:06.684Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-e4bc2fd4927d450a","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8b73bd0a025f4cdce5096c851e5e645","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:04:06.730Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-862220bf343a6df9f07f5619: done: 0.025904 sec","job_status":"done","queue_duration_s":5.157619,"scheduling_latency_s":5.157648,"redis_calls":10,"redis_duration_s":0.001997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025904,"completed_at":"2026-11-07T11:04:11.913Z","load_balancing_strategy":"primary","db_duration_s":0.002257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 363.313179] setup-gitlab-start[3559]: W, [2026-11-07T11:04:38.036338 #3559] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 388.917490] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:03.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6fe78da4ea1b1382db2a6537","created_at":"2026-11-07T11:05:03.631Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78823ba2f850f72a807d453d95b0a11c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:05:03.638Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6fe78da4ea1b1382db2a6537: start","job_status":"start","queue_duration_s":0.002141,"scheduling_latency_s":0.002186} vm-test-run-nixbot-gitlab> gitlab # [ 389.179680] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:03.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6fe78da4ea1b1382db2a6537","created_at":"2026-11-07T11:05:03.631Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78823ba2f850f72a807d453d95b0a11c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:05:03.638Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6fe78da4ea1b1382db2a6537: done: 0.262442 sec","job_status":"done","queue_duration_s":0.002141,"scheduling_latency_s":0.002186,"redis_calls":15,"redis_duration_s":0.063687,"redis_read_bytes":1029,"redis_write_bytes":1306,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.009246,"redis_feature_flag_read_bytes":1024,"redis_feature_flag_write_bytes":565,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.049349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.102,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.102,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.262442,"completed_at":"2026-11-07T11:05:03.902Z","load_balancing_strategy":"primary","db_duration_s":0.115929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.244621] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:03.934Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6cad1701e630a9a70f699710","created_at":"2026-11-07T11:05:03.656Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d7a50f5247dec925077b96e5750b48","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.760Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6cad1701e630a9a70f699710: start","job_status":"start","queue_duration_s":0.174129,"scheduling_latency_s":0.174212} vm-test-run-nixbot-gitlab> gitlab # [ 389.414462] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.137Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6cad1701e630a9a70f699710","created_at":"2026-11-07T11:05:03.656Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d7a50f5247dec925077b96e5750b48","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.760Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6cad1701e630a9a70f699710: done: 0.203154 sec","job_status":"done","queue_duration_s":0.174129,"scheduling_latency_s":0.174212,"redis_calls":11,"redis_duration_s":0.015546,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006262,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.203154,"completed_at":"2026-11-07T11:05:04.137Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.479849] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5eced1a6dd78ab14e9fa7c7d","created_at":"2026-11-07T11:05:03.773Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f39f587ed21f26b7a4a237035452603f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.781Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5eced1a6dd78ab14e9fa7c7d: start","job_status":"start","queue_duration_s":0.392068,"scheduling_latency_s":0.392117} vm-test-run-nixbot-gitlab> gitlab # [ 389.645348] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5eced1a6dd78ab14e9fa7c7d","created_at":"2026-11-07T11:05:03.773Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f39f587ed21f26b7a4a237035452603f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.781Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5eced1a6dd78ab14e9fa7c7d: done: 0.195062 sec","job_status":"done","queue_duration_s":0.392068,"scheduling_latency_s":0.392117,"redis_calls":10,"redis_duration_s":0.012977,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001861,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003646,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00213266,"exclusive_lock_hold_duration_s":0.03086618,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.195062,"completed_at":"2026-11-07T11:05:04.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.713203] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4713971d6435fbd2c036f023","created_at":"2026-11-07T11:05:03.787Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9eb480a3caf53ae2c147844372a3a3b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.793Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4713971d6435fbd2c036f023: start","job_status":"start","queue_duration_s":0.578072,"scheduling_latency_s":0.57812} vm-test-run-nixbot-gitlab> gitlab # [ 389.731507] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.393Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 389.736004] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4713971d6435fbd2c036f023","created_at":"2026-11-07T11:05:03.787Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9eb480a3caf53ae2c147844372a3a3b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.793Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4713971d6435fbd2c036f023: done: 0.031337 sec","job_status":"done","queue_duration_s":0.578072,"scheduling_latency_s":0.57812,"redis_calls":8,"redis_duration_s":0.0024010000000000004,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031337,"completed_at":"2026-11-07T11:05:04.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.783788] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b13593595593740b0f8a4de5","created_at":"2026-11-07T11:05:03.865Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42cde700b00c530fb36b2587aee79ac0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.878Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b13593595593740b0f8a4de5: start","job_status":"start","queue_duration_s":0.526943,"scheduling_latency_s":0.526976} vm-test-run-nixbot-gitlab> gitlab # [ 389.798389] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b13593595593740b0f8a4de5","created_at":"2026-11-07T11:05:03.865Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42cde700b00c530fb36b2587aee79ac0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.878Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b13593595593740b0f8a4de5: done: 0.036098 sec","job_status":"done","queue_duration_s":0.526943,"scheduling_latency_s":0.526976,"redis_calls":9,"redis_duration_s":0.002784,"redis_read_bytes":261,"redis_write_bytes":985,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":159,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.036098,"completed_at":"2026-11-07T11:05:04.441Z","load_balancing_strategy":"primary","db_duration_s":0.008292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.842509] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fb4729ccae1250e7e23f47c0","created_at":"2026-11-07T11:05:03.894Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9083c28e3cf9180c422a97ca35feb8d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.905Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fb4729ccae1250e7e23f47c0: start","job_status":"start","queue_duration_s":0.539878,"scheduling_latency_s":0.539916} vm-test-run-nixbot-gitlab> gitlab # [ 389.857531] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.461Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 389.860799] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.473Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fb4729ccae1250e7e23f47c0","created_at":"2026-11-07T11:05:03.894Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9083c28e3cf9180c422a97ca35feb8d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.905Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fb4729ccae1250e7e23f47c0: done: 0.028292 sec","job_status":"done","queue_duration_s":0.539878,"scheduling_latency_s":0.539916,"redis_calls":8,"redis_duration_s":0.002166,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000228,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028292,"completed_at":"2026-11-07T11:05:04.473Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.904290] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"827d7dc5671c449879053013","created_at":"2026-11-07T11:05:03.919Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"313ed105b9c035a31a0700dcff49bba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:05:03.928Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-827d7dc5671c449879053013: start","job_status":"start","queue_duration_s":0.548224,"scheduling_latency_s":0.548279} vm-test-run-nixbot-gitlab> gitlab # [ 389.918528] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.491Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 389.923499] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"827d7dc5671c449879053013","created_at":"2026-11-07T11:05:03.919Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"313ed105b9c035a31a0700dcff49bba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:05:03.928Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-827d7dc5671c449879053013: done: 0.028789 sec","job_status":"done","queue_duration_s":0.548224,"scheduling_latency_s":0.548279,"redis_calls":8,"redis_duration_s":0.002555,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000212,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028789,"completed_at":"2026-11-07T11:05:04.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 389.965879] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:04.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"09394cc4f2dee41d443261eb","created_at":"2026-11-07T11:05:03.933Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bba5196378ba9381319745a341d00626","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.942Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-09394cc4f2dee41d443261eb: start","job_status":"start","queue_duration_s":0.566385,"scheduling_latency_s":0.566432} vm-test-run-nixbot-gitlab> gitlab # [ 390.317656] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"09394cc4f2dee41d443261eb","created_at":"2026-11-07T11:05:03.933Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bba5196378ba9381319745a341d00626","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:03.942Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-09394cc4f2dee41d443261eb: done: 0.532531 sec","job_status":"done","queue_duration_s":0.566385,"scheduling_latency_s":0.566432,"redis_calls":563,"redis_duration_s":0.109427,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.107418,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.483816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.532531,"completed_at":"2026-11-07T11:05:05.040Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 390.385851] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c2e0450f494a3d2476f3c37a","created_at":"2026-11-07T11:05:03.965Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2d48d5de62536ba456f3d105fc8d31e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:05:03.971Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c2e0450f494a3d2476f3c37a: start","job_status":"start","queue_duration_s":1.071575,"scheduling_latency_s":1.071607} vm-test-run-nixbot-gitlab> gitlab # [ 390.404062] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.063Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 390.407942] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c2e0450f494a3d2476f3c37a","created_at":"2026-11-07T11:05:03.965Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2d48d5de62536ba456f3d105fc8d31e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:05:03.971Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c2e0450f494a3d2476f3c37a: done: 0.039752 sec","job_status":"done","queue_duration_s":1.071575,"scheduling_latency_s":1.071607,"redis_calls":8,"redis_duration_s":0.003407,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039752,"completed_at":"2026-11-07T11:05:05.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 390.451808] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"393c002dd971507e30768aed","created_at":"2026-11-07T11:05:04.007Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0e76205e802a546d6bd6c193d572491","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.014Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-393c002dd971507e30768aed: start","job_status":"start","queue_duration_s":1.072916,"scheduling_latency_s":1.072967} vm-test-run-nixbot-gitlab> gitlab # [ 390.467097] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"393c002dd971507e30768aed","created_at":"2026-11-07T11:05:04.007Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0e76205e802a546d6bd6c193d572491","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.014Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-393c002dd971507e30768aed: done: 0.063155 sec","job_status":"done","queue_duration_s":1.072916,"scheduling_latency_s":1.072967,"redis_calls":11,"redis_duration_s":0.005056,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002506,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01426094,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.063155,"completed_at":"2026-11-07T11:05:05.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005527,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 390.513419] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"af659900adca6159ca3bcec0","created_at":"2026-11-07T11:05:04.019Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f086f114e3849b0cb6ba52b0fcc0753","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.024Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-af659900adca6159ca3bcec0: start","job_status":"start","queue_duration_s":1.13158,"scheduling_latency_s":1.131621} vm-test-run-nixbot-gitlab> gitlab # [ 390.527447] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"af659900adca6159ca3bcec0","created_at":"2026-11-07T11:05:04.019Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f086f114e3849b0cb6ba52b0fcc0753","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.024Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-af659900adca6159ca3bcec0: done: 0.048404 sec","job_status":"done","queue_duration_s":1.13158,"scheduling_latency_s":1.131621,"redis_calls":10,"redis_duration_s":0.010092,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003636,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00123684,"exclusive_lock_hold_duration_s":0.00762886,"duration_s":0.048404,"completed_at":"2026-11-07T11:05:05.204Z","load_balancing_strategy":"primary","db_duration_s":0.002234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 390.571676] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.206Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6b8b56505f4af4818143b130","created_at":"2026-11-07T11:05:04.031Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28fd9cd0992dbc31e8672a64428b39f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.037Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6b8b56505f4af4818143b130: start","job_status":"start","queue_duration_s":1.169853,"scheduling_latency_s":1.169892} vm-test-run-nixbot-gitlab> gitlab # [ 390.830068] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6b8b56505f4af4818143b130","created_at":"2026-11-07T11:05:04.031Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28fd9cd0992dbc31e8672a64428b39f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.037Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6b8b56505f4af4818143b130: done: 0.346514 sec","job_status":"done","queue_duration_s":1.169853,"scheduling_latency_s":1.169892,"redis_calls":11,"redis_duration_s":0.00347,"redis_read_bytes":263,"redis_write_bytes":1138,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":151,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001572,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.295921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00088966,"exclusive_lock_hold_duration_s":0.31600808,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.346514,"completed_at":"2026-11-07T11:05:05.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 390.899617] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"21b5824dd2dd02c141b4e001","created_at":"2026-11-07T11:05:04.044Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.049Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-21b5824dd2dd02c141b4e001: start","job_status":"start","queue_duration_s":1.506265,"scheduling_latency_s":1.506299} vm-test-run-nixbot-gitlab> gitlab # [ 390.956884] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"21b5824dd2dd02c141b4e001","created_at":"2026-11-07T11:05:04.044Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.049Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-21b5824dd2dd02c141b4e001: done: 0.12383 sec","job_status":"done","queue_duration_s":1.506265,"scheduling_latency_s":1.506299,"redis_calls":65,"redis_duration_s":0.015717,"redis_read_bytes":748,"redis_write_bytes":41402,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.001113,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000789,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013061,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.12383,"completed_at":"2026-11-07T11:05:05.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.023577] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3590531526d268cc83b56d7d","created_at":"2026-11-07T11:05:04.068Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57461f82a551212e14cb2c0af01498c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.078Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3590531526d268cc83b56d7d: start","job_status":"start","queue_duration_s":1.60303,"scheduling_latency_s":1.603088} vm-test-run-nixbot-gitlab> gitlab # [ 391.041546] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.697Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 391.045408] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3590531526d268cc83b56d7d","created_at":"2026-11-07T11:05:04.068Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57461f82a551212e14cb2c0af01498c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.078Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3590531526d268cc83b56d7d: done: 0.027078 sec","job_status":"done","queue_duration_s":1.60303,"scheduling_latency_s":1.603088,"redis_calls":8,"redis_duration_s":0.00228,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000229,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001489,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027078,"completed_at":"2026-11-07T11:05:05.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.089643] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e2dcdd4d3b61ba4fc2eea3c1","created_at":"2026-11-07T11:05:04.099Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67e026365ea5f0b945cd05c72e68d96c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.107Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e2dcdd4d3b61ba4fc2eea3c1: start","job_status":"start","queue_duration_s":1.603418,"scheduling_latency_s":1.603481} vm-test-run-nixbot-gitlab> gitlab # [ 391.104790] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e2dcdd4d3b61ba4fc2eea3c1","created_at":"2026-11-07T11:05:04.099Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67e026365ea5f0b945cd05c72e68d96c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.107Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e2dcdd4d3b61ba4fc2eea3c1: done: 0.02607 sec","job_status":"done","queue_duration_s":1.603418,"scheduling_latency_s":1.603481,"redis_calls":9,"redis_duration_s":0.002421,"redis_read_bytes":261,"redis_write_bytes":1008,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":177,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02607,"completed_at":"2026-11-07T11:05:05.736Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.148273] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"dab0800b06d2f04c8e9cfbba","created_at":"2026-11-07T11:05:04.123Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bc5896626bcd0721d2d4b0c1e10a9c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.142Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-dab0800b06d2f04c8e9cfbba: start","job_status":"start","queue_duration_s":1.596477,"scheduling_latency_s":1.596514} vm-test-run-nixbot-gitlab> gitlab # [ 391.163952] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"dab0800b06d2f04c8e9cfbba","created_at":"2026-11-07T11:05:04.123Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bc5896626bcd0721d2d4b0c1e10a9c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.142Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-dab0800b06d2f04c8e9cfbba: done: 0.045536 sec","job_status":"done","queue_duration_s":1.596477,"scheduling_latency_s":1.596514,"redis_calls":11,"redis_duration_s":0.00525,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000737,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000536,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00149742,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.045536,"completed_at":"2026-11-07T11:05:05.784Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.210014] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c486637ab807dd2d80efdb1","created_at":"2026-11-07T11:05:04.156Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e45ba86fc9eaf83b770abb1d4afe381c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.164Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c486637ab807dd2d80efdb1: start","job_status":"start","queue_duration_s":1.626624,"scheduling_latency_s":1.626672} vm-test-run-nixbot-gitlab> gitlab # [ 391.225218] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.809Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 391.229120] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c486637ab807dd2d80efdb1","created_at":"2026-11-07T11:05:04.156Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e45ba86fc9eaf83b770abb1d4afe381c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.164Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c486637ab807dd2d80efdb1: done: 0.030138 sec","job_status":"done","queue_duration_s":1.626624,"scheduling_latency_s":1.626672,"redis_calls":8,"redis_duration_s":0.002487,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001587,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030138,"completed_at":"2026-11-07T11:05:05.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.272879] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"61910de20bfa58bde2f88d14","created_at":"2026-11-07T11:05:04.172Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e0f6f452fe90b3047f6e42721a15877","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.181Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-61910de20bfa58bde2f88d14: start","job_status":"start","queue_duration_s":1.643362,"scheduling_latency_s":1.643409} vm-test-run-nixbot-gitlab> gitlab # [ 391.287431] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"61910de20bfa58bde2f88d14","created_at":"2026-11-07T11:05:04.172Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e0f6f452fe90b3047f6e42721a15877","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.181Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-61910de20bfa58bde2f88d14: done: 0.037756 sec","job_status":"done","queue_duration_s":1.643362,"scheduling_latency_s":1.643409,"redis_calls":8,"redis_duration_s":0.0030719999999999996,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001969,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037756,"completed_at":"2026-11-07T11:05:05.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.331265] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e86e5b75064389b74a4f9f06","created_at":"2026-11-07T11:05:04.187Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6107286fc991cfc3ba96c283dee606de","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.193Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e86e5b75064389b74a4f9f06: start","job_status":"start","queue_duration_s":1.671381,"scheduling_latency_s":1.671424} vm-test-run-nixbot-gitlab> gitlab # [ 391.346252] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e86e5b75064389b74a4f9f06","created_at":"2026-11-07T11:05:04.187Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6107286fc991cfc3ba96c283dee606de","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.193Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e86e5b75064389b74a4f9f06: done: 0.029764 sec","job_status":"done","queue_duration_s":1.671381,"scheduling_latency_s":1.671424,"redis_calls":10,"redis_duration_s":0.0031450000000000002,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.0002,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008156,"exclusive_lock_hold_duration_s":0.00476004,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029764,"completed_at":"2026-11-07T11:05:05.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.392269] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a9a9e572b5bf102a2b5f72ec","created_at":"2026-11-07T11:05:04.236Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db3dded741a668511ceb6aa3fde219dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.244Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a9a9e572b5bf102a2b5f72ec: start","job_status":"start","queue_duration_s":1.65259,"scheduling_latency_s":1.652622} vm-test-run-nixbot-gitlab> gitlab # [ 391.407313] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a9a9e572b5bf102a2b5f72ec","created_at":"2026-11-07T11:05:04.236Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db3dded741a668511ceb6aa3fde219dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.244Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a9a9e572b5bf102a2b5f72ec: done: 0.01996 sec","job_status":"done","queue_duration_s":1.65259,"scheduling_latency_s":1.652622,"redis_calls":8,"redis_duration_s":0.0018540000000000002,"redis_read_bytes":133,"redis_write_bytes":879,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000196,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":103,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01996,"completed_at":"2026-11-07T11:05:05.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.451304] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4b2e6f54656a09062bb44684","created_at":"2026-11-07T11:05:04.251Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"080bfed73628c03df435eba3fab9a1b2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.257Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4b2e6f54656a09062bb44684: start","job_status":"start","queue_duration_s":1.661225,"scheduling_latency_s":1.661281} vm-test-run-nixbot-gitlab> gitlab # [ 391.466291] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.970Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4b2e6f54656a09062bb44684","created_at":"2026-11-07T11:05:04.251Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"080bfed73628c03df435eba3fab9a1b2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.257Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4b2e6f54656a09062bb44684: done: 0.051717 sec","job_status":"done","queue_duration_s":1.661225,"scheduling_latency_s":1.661281,"redis_calls":10,"redis_duration_s":0.0028810000000000003,"redis_read_bytes":135,"redis_write_bytes":1267,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000209,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":105,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001335,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008433,"exclusive_lock_hold_duration_s":0.02910218,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.051717,"completed_at":"2026-11-07T11:05:05.970Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.513066] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:05.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"926d005051ff8912f9f004c9","created_at":"2026-11-07T11:05:04.274Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe7e5bdf1c3b98e6d81322305dc697e1","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.293Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-926d005051ff8912f9f004c9: start","job_status":"start","queue_duration_s":1.678935,"scheduling_latency_s":1.678969} vm-test-run-nixbot-gitlab> gitlab # [ 391.526817] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"926d005051ff8912f9f004c9","created_at":"2026-11-07T11:05:04.274Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe7e5bdf1c3b98e6d81322305dc697e1","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.293Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-926d005051ff8912f9f004c9: done: 0.053922 sec","job_status":"done","queue_duration_s":1.678935,"scheduling_latency_s":1.678969,"redis_calls":10,"redis_duration_s":0.003708,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001168,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054746,"duration_s":0.053922,"completed_at":"2026-11-07T11:05:06.026Z","load_balancing_strategy":"primary","db_duration_s":0.008885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.570606] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d79e605b96b92ed8a2789ed5","created_at":"2026-11-07T11:05:04.312Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.319Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d79e605b96b92ed8a2789ed5: start","job_status":"start","queue_duration_s":1.709829,"scheduling_latency_s":1.709874} vm-test-run-nixbot-gitlab> gitlab # [ 391.584641] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d79e605b96b92ed8a2789ed5","created_at":"2026-11-07T11:05:04.312Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-c912a39a4c6e4c90","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:04.319Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d79e605b96b92ed8a2789ed5: done: 0.052891 sec","job_status":"done","queue_duration_s":1.709829,"scheduling_latency_s":1.709874,"redis_calls":16,"redis_duration_s":0.004953,"redis_read_bytes":272,"redis_write_bytes":4217,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000569,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.000597,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3007,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002675,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052891,"completed_at":"2026-11-07T11:05:06.082Z","load_balancing_strategy":"primary","db_duration_s":0.001929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.628249] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcdb60c1e8c0711e1a194bf4","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcdb60c1e8c0711e1a194bf4: start","job_status":"start","queue_duration_s":0.416755,"scheduling_latency_s":0.41679} vm-test-run-nixbot-gitlab> gitlab # [ 391.641035] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcdb60c1e8c0711e1a194bf4","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcdb60c1e8c0711e1a194bf4: done: 0.050366 sec","job_status":"done","queue_duration_s":0.416755,"scheduling_latency_s":0.41679,"redis_calls":14,"redis_duration_s":0.018858,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002798,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050366,"completed_at":"2026-11-07T11:05:06.134Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.682967] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54de87fce112d62f131291a9","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54de87fce112d62f131291a9: start","job_status":"start","queue_duration_s":0.469419,"scheduling_latency_s":0.469455} vm-test-run-nixbot-gitlab> gitlab # [ 391.695966] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54de87fce112d62f131291a9","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54de87fce112d62f131291a9: done: 0.026389 sec","job_status":"done","queue_duration_s":0.469419,"scheduling_latency_s":0.469455,"redis_calls":13,"redis_duration_s":0.0029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001748,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026389,"completed_at":"2026-11-07T11:05:06.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.736916] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db80997fb8920c464ee7f962","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db80997fb8920c464ee7f962: start","job_status":"start","queue_duration_s":0.498009,"scheduling_latency_s":0.49805} vm-test-run-nixbot-gitlab> gitlab # [ 391.749690] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db80997fb8920c464ee7f962","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db80997fb8920c464ee7f962: done: 0.027685 sec","job_status":"done","queue_duration_s":0.498009,"scheduling_latency_s":0.49805,"redis_calls":13,"redis_duration_s":0.003487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027685,"completed_at":"2026-11-07T11:05:06.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.791266] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ecf7d2a5be17e2b5c39650","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ecf7d2a5be17e2b5c39650: start","job_status":"start","queue_duration_s":0.527723,"scheduling_latency_s":0.52776} vm-test-run-nixbot-gitlab> gitlab # [ 391.804401] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ecf7d2a5be17e2b5c39650","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ecf7d2a5be17e2b5c39650: done: 0.02612 sec","job_status":"done","queue_duration_s":0.527723,"scheduling_latency_s":0.52776,"redis_calls":13,"redis_duration_s":0.0029449999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02612,"completed_at":"2026-11-07T11:05:06.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.845318] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30d3b19f21062bd9e7d209c8","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30d3b19f21062bd9e7d209c8: start","job_status":"start","queue_duration_s":0.556044,"scheduling_latency_s":0.556079} vm-test-run-nixbot-gitlab> gitlab # [ 391.858404] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30d3b19f21062bd9e7d209c8","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30d3b19f21062bd9e7d209c8: done: 0.02572 sec","job_status":"done","queue_duration_s":0.556044,"scheduling_latency_s":0.556079,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02572,"completed_at":"2026-11-07T11:05:06.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.898624] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"561d5ce95fb18523f698cad5","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-561d5ce95fb18523f698cad5: start","job_status":"start","queue_duration_s":0.583758,"scheduling_latency_s":0.583794} vm-test-run-nixbot-gitlab> gitlab # [ 391.911631] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"561d5ce95fb18523f698cad5","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-561d5ce95fb18523f698cad5: done: 0.026352 sec","job_status":"done","queue_duration_s":0.583758,"scheduling_latency_s":0.583794,"redis_calls":13,"redis_duration_s":0.002752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026352,"completed_at":"2026-11-07T11:05:06.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 391.952472] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"406a3201f6926cc96f9c31ea","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-406a3201f6926cc96f9c31ea: start","job_status":"start","queue_duration_s":0.612141,"scheduling_latency_s":0.612177} vm-test-run-nixbot-gitlab> gitlab # [ 391.965681] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"406a3201f6926cc96f9c31ea","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-406a3201f6926cc96f9c31ea: done: 0.019967 sec","job_status":"done","queue_duration_s":0.612141,"scheduling_latency_s":0.612177,"redis_calls":13,"redis_duration_s":0.0018009999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019967,"completed_at":"2026-11-07T11:05:06.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.007209] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa626b622078ef85321ad475","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa626b622078ef85321ad475: start","job_status":"start","queue_duration_s":0.633658,"scheduling_latency_s":0.633685} vm-test-run-nixbot-gitlab> gitlab # [ 392.020206] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa626b622078ef85321ad475","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa626b622078ef85321ad475: done: 0.019013 sec","job_status":"done","queue_duration_s":0.633658,"scheduling_latency_s":0.633685,"redis_calls":13,"redis_duration_s":0.001975,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001324,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019013,"completed_at":"2026-11-07T11:05:06.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.060575] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91718211e1847aa7ed41ad9e","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91718211e1847aa7ed41ad9e: start","job_status":"start","queue_duration_s":0.654423,"scheduling_latency_s":0.654452} vm-test-run-nixbot-gitlab> gitlab # [ 392.073322] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91718211e1847aa7ed41ad9e","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91718211e1847aa7ed41ad9e: done: 0.020045 sec","job_status":"done","queue_duration_s":0.654423,"scheduling_latency_s":0.654452,"redis_calls":13,"redis_duration_s":0.002129,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001341,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020045,"completed_at":"2026-11-07T11:05:06.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.113796] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1db86eb71ab4cb0ee025b2cb","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1db86eb71ab4cb0ee025b2cb: start","job_status":"start","queue_duration_s":0.676171,"scheduling_latency_s":0.676199} vm-test-run-nixbot-gitlab> gitlab # [ 392.126474] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1db86eb71ab4cb0ee025b2cb","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1db86eb71ab4cb0ee025b2cb: done: 0.019762 sec","job_status":"done","queue_duration_s":0.676171,"scheduling_latency_s":0.676199,"redis_calls":13,"redis_duration_s":0.002041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019762,"completed_at":"2026-11-07T11:05:06.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.166931] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7357523a4b9c775bb815383f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7357523a4b9c775bb815383f: start","job_status":"start","queue_duration_s":0.698013,"scheduling_latency_s":0.698042} vm-test-run-nixbot-gitlab> gitlab # [ 392.180766] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7357523a4b9c775bb815383f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7357523a4b9c775bb815383f: done: 0.02289 sec","job_status":"done","queue_duration_s":0.698013,"scheduling_latency_s":0.698042,"redis_calls":13,"redis_duration_s":0.002421,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02289,"completed_at":"2026-11-07T11:05:06.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.220966] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51135cec648f35f2edf155b2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51135cec648f35f2edf155b2: start","job_status":"start","queue_duration_s":0.722692,"scheduling_latency_s":0.722724} vm-test-run-nixbot-gitlab> gitlab # [ 392.233841] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51135cec648f35f2edf155b2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51135cec648f35f2edf155b2: done: 0.023804 sec","job_status":"done","queue_duration_s":0.722692,"scheduling_latency_s":0.722724,"redis_calls":13,"redis_duration_s":0.0025729999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023804,"completed_at":"2026-11-07T11:05:06.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.274653] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d18c89e645745ad16e574a6e","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d18c89e645745ad16e574a6e: start","job_status":"start","queue_duration_s":0.748375,"scheduling_latency_s":0.748453} vm-test-run-nixbot-gitlab> gitlab # [ 392.287613] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d18c89e645745ad16e574a6e","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d18c89e645745ad16e574a6e: done: 0.022695 sec","job_status":"done","queue_duration_s":0.748375,"scheduling_latency_s":0.748453,"redis_calls":13,"redis_duration_s":0.002356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022695,"completed_at":"2026-11-07T11:05:06.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.328158] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d66e56b3ece6640785f4b93","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d66e56b3ece6640785f4b93: start","job_status":"start","queue_duration_s":0.772932,"scheduling_latency_s":0.772964} vm-test-run-nixbot-gitlab> gitlab # [ 392.340808] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d66e56b3ece6640785f4b93","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d66e56b3ece6640785f4b93: done: 0.023183 sec","job_status":"done","queue_duration_s":0.772932,"scheduling_latency_s":0.772964,"redis_calls":13,"redis_duration_s":0.0023680000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023183,"completed_at":"2026-11-07T11:05:06.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.381729] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5765fb4e171942af183d4a00","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5765fb4e171942af183d4a00: start","job_status":"start","queue_duration_s":0.798153,"scheduling_latency_s":0.798186} vm-test-run-nixbot-gitlab> gitlab # [ 392.394577] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5765fb4e171942af183d4a00","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5765fb4e171942af183d4a00: done: 0.022898 sec","job_status":"done","queue_duration_s":0.798153,"scheduling_latency_s":0.798186,"redis_calls":13,"redis_duration_s":0.00238,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001522,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022898,"completed_at":"2026-11-07T11:05:06.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.435307] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60551d3d5e16aaa515c99d3b","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60551d3d5e16aaa515c99d3b: start","job_status":"start","queue_duration_s":0.822802,"scheduling_latency_s":0.822833} vm-test-run-nixbot-gitlab> gitlab # [ 392.448865] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60551d3d5e16aaa515c99d3b","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60551d3d5e16aaa515c99d3b: done: 0.022684 sec","job_status":"done","queue_duration_s":0.822802,"scheduling_latency_s":0.822833,"redis_calls":13,"redis_duration_s":0.002218,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001413,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022684,"completed_at":"2026-11-07T11:05:06.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.489481] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c436bc81586776c45855be9f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c436bc81586776c45855be9f: start","job_status":"start","queue_duration_s":0.847322,"scheduling_latency_s":0.84739} vm-test-run-nixbot-gitlab> gitlab # [ 392.502232] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c436bc81586776c45855be9f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c436bc81586776c45855be9f: done: 0.022646 sec","job_status":"done","queue_duration_s":0.847322,"scheduling_latency_s":0.84739,"redis_calls":13,"redis_duration_s":0.002467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022646,"completed_at":"2026-11-07T11:05:06.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.542613] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2190f00373ec9914f04ba9d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2190f00373ec9914f04ba9d: start","job_status":"start","queue_duration_s":0.871767,"scheduling_latency_s":0.871799} vm-test-run-nixbot-gitlab> gitlab # [ 392.555861] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2190f00373ec9914f04ba9d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2190f00373ec9914f04ba9d: done: 0.023237 sec","job_status":"done","queue_duration_s":0.871767,"scheduling_latency_s":0.871799,"redis_calls":13,"redis_duration_s":0.002721,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023237,"completed_at":"2026-11-07T11:05:06.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.597720] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b64689428dfa90c1a8594327","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b64689428dfa90c1a8594327: start","job_status":"start","queue_duration_s":0.897138,"scheduling_latency_s":0.897172} vm-test-run-nixbot-gitlab> gitlab # [ 392.610688] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b64689428dfa90c1a8594327","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b64689428dfa90c1a8594327: done: 0.02331 sec","job_status":"done","queue_duration_s":0.897138,"scheduling_latency_s":0.897172,"redis_calls":13,"redis_duration_s":0.0023610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02331,"completed_at":"2026-11-07T11:05:06.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.651159] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccdf519a6f88311ba65b9e43","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccdf519a6f88311ba65b9e43: start","job_status":"start","queue_duration_s":0.922389,"scheduling_latency_s":0.922422} vm-test-run-nixbot-gitlab> gitlab # [ 392.663977] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccdf519a6f88311ba65b9e43","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccdf519a6f88311ba65b9e43: done: 0.022196 sec","job_status":"done","queue_duration_s":0.922389,"scheduling_latency_s":0.922422,"redis_calls":13,"redis_duration_s":0.002144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001367,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022196,"completed_at":"2026-11-07T11:05:06.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.704527] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bc35f87f82af03f1585566","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bc35f87f82af03f1585566: start","job_status":"start","queue_duration_s":0.946468,"scheduling_latency_s":0.946499} vm-test-run-nixbot-gitlab> gitlab # [ 392.717169] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99bc35f87f82af03f1585566","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99bc35f87f82af03f1585566: done: 0.022376 sec","job_status":"done","queue_duration_s":0.946468,"scheduling_latency_s":0.946499,"redis_calls":13,"redis_duration_s":0.002435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022376,"completed_at":"2026-11-07T11:05:06.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.757418] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd16b97c6c596a6c8449188d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd16b97c6c596a6c8449188d: start","job_status":"start","queue_duration_s":0.970674,"scheduling_latency_s":0.970707} vm-test-run-nixbot-gitlab> gitlab # [ 392.770164] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd16b97c6c596a6c8449188d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd16b97c6c596a6c8449188d: done: 0.022811 sec","job_status":"done","queue_duration_s":0.970674,"scheduling_latency_s":0.970707,"redis_calls":13,"redis_duration_s":0.002676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022811,"completed_at":"2026-11-07T11:05:06.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.811546] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8246fcbc7a129bfa5ccd2028","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8246fcbc7a129bfa5ccd2028: start","job_status":"start","queue_duration_s":0.995491,"scheduling_latency_s":0.995527} vm-test-run-nixbot-gitlab> gitlab # [ 392.824525] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8246fcbc7a129bfa5ccd2028","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8246fcbc7a129bfa5ccd2028: done: 0.023071 sec","job_status":"done","queue_duration_s":0.995491,"scheduling_latency_s":0.995527,"redis_calls":13,"redis_duration_s":0.002478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023071,"completed_at":"2026-11-07T11:05:06.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.864537] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccd511392807297f3d24797d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccd511392807297f3d24797d: start","job_status":"start","queue_duration_s":1.020334,"scheduling_latency_s":1.020365} vm-test-run-nixbot-gitlab> gitlab # [ 392.877174] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccd511392807297f3d24797d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccd511392807297f3d24797d: done: 0.022374 sec","job_status":"done","queue_duration_s":1.020334,"scheduling_latency_s":1.020365,"redis_calls":13,"redis_duration_s":0.0021,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022374,"completed_at":"2026-11-07T11:05:06.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.917442] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0c90d191c39d00cb21b678","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0c90d191c39d00cb21b678: start","job_status":"start","queue_duration_s":1.044505,"scheduling_latency_s":1.044536} vm-test-run-nixbot-gitlab> gitlab # [ 392.930231] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0c90d191c39d00cb21b678","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0c90d191c39d00cb21b678: done: 0.021858 sec","job_status":"done","queue_duration_s":1.044505,"scheduling_latency_s":1.044536,"redis_calls":13,"redis_duration_s":0.002297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001429,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021858,"completed_at":"2026-11-07T11:05:06.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 392.971143] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f288783847d283939306c2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f288783847d283939306c2: start","job_status":"start","queue_duration_s":1.0681,"scheduling_latency_s":1.068129} vm-test-run-nixbot-gitlab> gitlab # [ 392.984149] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f288783847d283939306c2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f288783847d283939306c2: done: 0.0192 sec","job_status":"done","queue_duration_s":1.0681,"scheduling_latency_s":1.068129,"redis_calls":13,"redis_duration_s":0.001786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0192,"completed_at":"2026-11-07T11:05:06.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.025280] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"122ef6bb7a991be2e5cea322","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-122ef6bb7a991be2e5cea322: start","job_status":"start","queue_duration_s":1.088806,"scheduling_latency_s":1.088832} vm-test-run-nixbot-gitlab> gitlab # [ 393.038011] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"122ef6bb7a991be2e5cea322","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-122ef6bb7a991be2e5cea322: done: 0.019356 sec","job_status":"done","queue_duration_s":1.088806,"scheduling_latency_s":1.088832,"redis_calls":13,"redis_duration_s":0.002598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019356,"completed_at":"2026-11-07T11:05:06.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.078270] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0373f3b7648372576b35d8ba","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0373f3b7648372576b35d8ba: start","job_status":"start","queue_duration_s":1.109799,"scheduling_latency_s":1.109827} vm-test-run-nixbot-gitlab> gitlab # [ 393.091083] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0373f3b7648372576b35d8ba","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0373f3b7648372576b35d8ba: done: 0.019951 sec","job_status":"done","queue_duration_s":1.109799,"scheduling_latency_s":1.109827,"redis_calls":13,"redis_duration_s":0.002547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001367,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019951,"completed_at":"2026-11-07T11:05:06.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.131637] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6930034a1b23b70abeabcf5","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6930034a1b23b70abeabcf5: start","job_status":"start","queue_duration_s":1.131257,"scheduling_latency_s":1.131282} vm-test-run-nixbot-gitlab> gitlab # [ 393.144654] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6930034a1b23b70abeabcf5","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6930034a1b23b70abeabcf5: done: 0.018984 sec","job_status":"done","queue_duration_s":1.131257,"scheduling_latency_s":1.131282,"redis_calls":13,"redis_duration_s":0.0018510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018984,"completed_at":"2026-11-07T11:05:06.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.185345] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"112055134e147a09d1bd8f43","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-112055134e147a09d1bd8f43: start","job_status":"start","queue_duration_s":1.152602,"scheduling_latency_s":1.152631} vm-test-run-nixbot-gitlab> gitlab # [ 393.198164] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"112055134e147a09d1bd8f43","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-112055134e147a09d1bd8f43: done: 0.037152 sec","job_status":"done","queue_duration_s":1.152602,"scheduling_latency_s":1.152631,"redis_calls":13,"redis_duration_s":0.002399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001385,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037152,"completed_at":"2026-11-07T11:05:06.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.239417] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69f689f67678f97d43b5c1ef","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69f689f67678f97d43b5c1ef: start","job_status":"start","queue_duration_s":1.191473,"scheduling_latency_s":1.191504} vm-test-run-nixbot-gitlab> gitlab # [ 393.252384] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69f689f67678f97d43b5c1ef","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69f689f67678f97d43b5c1ef: done: 0.019286 sec","job_status":"done","queue_duration_s":1.191473,"scheduling_latency_s":1.191504,"redis_calls":13,"redis_duration_s":0.002058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019286,"completed_at":"2026-11-07T11:05:06.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.292919] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adebfb8752bfa1f1c162cbbb","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adebfb8752bfa1f1c162cbbb: start","job_status":"start","queue_duration_s":1.212493,"scheduling_latency_s":1.212525} vm-test-run-nixbot-gitlab> gitlab # [ 393.305653] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adebfb8752bfa1f1c162cbbb","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adebfb8752bfa1f1c162cbbb: done: 0.020003 sec","job_status":"done","queue_duration_s":1.212493,"scheduling_latency_s":1.212525,"redis_calls":13,"redis_duration_s":0.002535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020003,"completed_at":"2026-11-07T11:05:06.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.345880] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eb91f32997cce75d7117dd2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eb91f32997cce75d7117dd2: start","job_status":"start","queue_duration_s":1.23406,"scheduling_latency_s":1.234087} vm-test-run-nixbot-gitlab> gitlab # [ 393.358686] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eb91f32997cce75d7117dd2","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eb91f32997cce75d7117dd2: done: 0.02047 sec","job_status":"done","queue_duration_s":1.23406,"scheduling_latency_s":1.234087,"redis_calls":13,"redis_duration_s":0.002491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001537,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02047,"completed_at":"2026-11-07T11:05:06.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.399029] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5dbfb2163f4afbf7baa036","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5dbfb2163f4afbf7baa036: start","job_status":"start","queue_duration_s":1.256218,"scheduling_latency_s":1.256246} vm-test-run-nixbot-gitlab> gitlab # [ 393.412945] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5dbfb2163f4afbf7baa036","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5dbfb2163f4afbf7baa036: done: 0.019669 sec","job_status":"done","queue_duration_s":1.256218,"scheduling_latency_s":1.256246,"redis_calls":13,"redis_duration_s":0.002394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019669,"completed_at":"2026-11-07T11:05:06.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.453170] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0586c8128031b8d615092b14","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0586c8128031b8d615092b14: start","job_status":"start","queue_duration_s":1.277484,"scheduling_latency_s":1.277509} vm-test-run-nixbot-gitlab> gitlab # [ 393.465801] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0586c8128031b8d615092b14","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0586c8128031b8d615092b14: done: 0.019452 sec","job_status":"done","queue_duration_s":1.277484,"scheduling_latency_s":1.277509,"redis_calls":13,"redis_duration_s":0.002103,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019452,"completed_at":"2026-11-07T11:05:06.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.506833] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f547e175400e98abf8ecccf0","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f547e175400e98abf8ecccf0: start","job_status":"start","queue_duration_s":1.298596,"scheduling_latency_s":1.298624} vm-test-run-nixbot-gitlab> gitlab # [ 393.519715] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f547e175400e98abf8ecccf0","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f547e175400e98abf8ecccf0: done: 0.02015 sec","job_status":"done","queue_duration_s":1.298596,"scheduling_latency_s":1.298624,"redis_calls":13,"redis_duration_s":0.002282,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02015,"completed_at":"2026-11-07T11:05:06.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.559958] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:06.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c862f2493017933d8f2a7dde","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c862f2493017933d8f2a7dde: start","job_status":"start","queue_duration_s":1.32064,"scheduling_latency_s":1.320667} vm-test-run-nixbot-gitlab> gitlab # [ 393.572884] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c862f2493017933d8f2a7dde","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c862f2493017933d8f2a7dde: done: 0.020049 sec","job_status":"done","queue_duration_s":1.32064,"scheduling_latency_s":1.320667,"redis_calls":13,"redis_duration_s":0.002197,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001387,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020049,"completed_at":"2026-11-07T11:05:07.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.614162] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"708403a03787e55122ff546f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-708403a03787e55122ff546f: start","job_status":"start","queue_duration_s":1.342289,"scheduling_latency_s":1.342316} vm-test-run-nixbot-gitlab> gitlab # [ 393.626950] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"708403a03787e55122ff546f","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-708403a03787e55122ff546f: done: 0.020717 sec","job_status":"done","queue_duration_s":1.342289,"scheduling_latency_s":1.342316,"redis_calls":13,"redis_duration_s":0.0023309999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020717,"completed_at":"2026-11-07T11:05:07.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.667406] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9a90a9a28dbd334e3bf6dc0","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9a90a9a28dbd334e3bf6dc0: start","job_status":"start","queue_duration_s":1.364689,"scheduling_latency_s":1.364717} vm-test-run-nixbot-gitlab> gitlab # [ 393.680284] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9a90a9a28dbd334e3bf6dc0","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9a90a9a28dbd334e3bf6dc0: done: 0.018876 sec","job_status":"done","queue_duration_s":1.364689,"scheduling_latency_s":1.364717,"redis_calls":13,"redis_duration_s":0.002006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018876,"completed_at":"2026-11-07T11:05:07.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.720680] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9961711112f720117616439","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9961711112f720117616439: start","job_status":"start","queue_duration_s":1.38521,"scheduling_latency_s":1.385238} vm-test-run-nixbot-gitlab> gitlab # [ 393.733343] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9961711112f720117616439","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9961711112f720117616439: done: 0.020531 sec","job_status":"done","queue_duration_s":1.38521,"scheduling_latency_s":1.385238,"redis_calls":13,"redis_duration_s":0.002484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020531,"completed_at":"2026-11-07T11:05:07.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.773639] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c64b1d3365193fd71d58473","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c64b1d3365193fd71d58473: start","job_status":"start","queue_duration_s":1.407302,"scheduling_latency_s":1.40743} vm-test-run-nixbot-gitlab> gitlab # [ 393.786467] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c64b1d3365193fd71d58473","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c64b1d3365193fd71d58473: done: 0.019433 sec","job_status":"done","queue_duration_s":1.407302,"scheduling_latency_s":1.40743,"redis_calls":13,"redis_duration_s":0.002044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019433,"completed_at":"2026-11-07T11:05:07.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.827981] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9064fb667a0dedc8c1fb9660","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9064fb667a0dedc8c1fb9660: start","job_status":"start","queue_duration_s":1.42828,"scheduling_latency_s":1.428307} vm-test-run-nixbot-gitlab> gitlab # [ 393.840711] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9064fb667a0dedc8c1fb9660","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9064fb667a0dedc8c1fb9660: done: 0.019386 sec","job_status":"done","queue_duration_s":1.42828,"scheduling_latency_s":1.428307,"redis_calls":13,"redis_duration_s":0.002029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019386,"completed_at":"2026-11-07T11:05:07.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.881178] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0549141022034c11e0487e7d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0549141022034c11e0487e7d: start","job_status":"start","queue_duration_s":1.449255,"scheduling_latency_s":1.449281} vm-test-run-nixbot-gitlab> gitlab # [ 393.893958] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0549141022034c11e0487e7d","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0549141022034c11e0487e7d: done: 0.019291 sec","job_status":"done","queue_duration_s":1.449255,"scheduling_latency_s":1.449281,"redis_calls":13,"redis_duration_s":0.001988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019291,"completed_at":"2026-11-07T11:05:07.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.934581] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98b911ef455d05f1e22ebd97","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98b911ef455d05f1e22ebd97: start","job_status":"start","queue_duration_s":1.469992,"scheduling_latency_s":1.470016} vm-test-run-nixbot-gitlab> gitlab # [ 393.947297] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98b911ef455d05f1e22ebd97","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98b911ef455d05f1e22ebd97: done: 0.01924 sec","job_status":"done","queue_duration_s":1.469992,"scheduling_latency_s":1.470016,"redis_calls":13,"redis_duration_s":0.002137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001371,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01924,"completed_at":"2026-11-07T11:05:07.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 393.990146] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57eae76e5d29fd971237e466","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57eae76e5d29fd971237e466: start","job_status":"start","queue_duration_s":1.490814,"scheduling_latency_s":1.490839} vm-test-run-nixbot-gitlab> gitlab # [ 394.003174] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57eae76e5d29fd971237e466","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57eae76e5d29fd971237e466: done: 0.020245 sec","job_status":"done","queue_duration_s":1.490814,"scheduling_latency_s":1.490839,"redis_calls":13,"redis_duration_s":0.0024,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020245,"completed_at":"2026-11-07T11:05:07.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.044982] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4309898e6164db15e33a688","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4309898e6164db15e33a688: start","job_status":"start","queue_duration_s":1.512738,"scheduling_latency_s":1.512765} vm-test-run-nixbot-gitlab> gitlab # [ 394.057551] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4309898e6164db15e33a688","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4309898e6164db15e33a688: done: 0.019548 sec","job_status":"done","queue_duration_s":1.512738,"scheduling_latency_s":1.512765,"redis_calls":13,"redis_duration_s":0.002049,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019548,"completed_at":"2026-11-07T11:05:07.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.097911] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40514ca2954a66a0ab2c8654","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40514ca2954a66a0ab2c8654: start","job_status":"start","queue_duration_s":1.534083,"scheduling_latency_s":1.53411} vm-test-run-nixbot-gitlab> gitlab # [ 394.110726] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40514ca2954a66a0ab2c8654","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40514ca2954a66a0ab2c8654: done: 0.018846 sec","job_status":"done","queue_duration_s":1.534083,"scheduling_latency_s":1.53411,"redis_calls":13,"redis_duration_s":0.0019329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018846,"completed_at":"2026-11-07T11:05:07.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.151468] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb23c4dd881abfc1a8aa8588","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb23c4dd881abfc1a8aa8588: start","job_status":"start","queue_duration_s":1.554456,"scheduling_latency_s":1.554483} vm-test-run-nixbot-gitlab> gitlab # [ 394.165253] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb23c4dd881abfc1a8aa8588","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb23c4dd881abfc1a8aa8588: done: 0.01929 sec","job_status":"done","queue_duration_s":1.554456,"scheduling_latency_s":1.554483,"redis_calls":13,"redis_duration_s":0.002002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001284,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01929,"completed_at":"2026-11-07T11:05:07.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.206100] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d00bc27e4eabdd242a72af7","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d00bc27e4eabdd242a72af7: start","job_status":"start","queue_duration_s":1.575284,"scheduling_latency_s":1.57531} vm-test-run-nixbot-gitlab> gitlab # [ 394.219057] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d00bc27e4eabdd242a72af7","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d00bc27e4eabdd242a72af7: done: 0.018912 sec","job_status":"done","queue_duration_s":1.575284,"scheduling_latency_s":1.57531,"redis_calls":13,"redis_duration_s":0.002007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018912,"completed_at":"2026-11-07T11:05:07.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.260881] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96e284813f462dff944b16b6","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96e284813f462dff944b16b6: start","job_status":"start","queue_duration_s":1.595785,"scheduling_latency_s":1.595811} vm-test-run-nixbot-gitlab> gitlab # [ 394.273689] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96e284813f462dff944b16b6","created_at":"2026-11-07T11:05:05.573Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-a5ca47deadc842e5","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4c51cebea7e0c3861834787958737977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:05.667Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96e284813f462dff944b16b6: done: 0.019796 sec","job_status":"done","queue_duration_s":1.595785,"scheduling_latency_s":1.595811,"redis_calls":13,"redis_duration_s":0.002177,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019796,"completed_at":"2026-11-07T11:05:07.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.314099] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.285Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"612545ed094207c1e7ac413f","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-612545ed094207c1e7ac413f: start","job_status":"start","queue_duration_s":1.213306,"scheduling_latency_s":1.213334} vm-test-run-nixbot-gitlab> gitlab # [ 394.326806] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.302Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 394.328570] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.314Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"612545ed094207c1e7ac413f","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-612545ed094207c1e7ac413f: done: 0.029493 sec","job_status":"done","queue_duration_s":1.213306,"scheduling_latency_s":1.213334,"redis_calls":11,"redis_duration_s":0.002283,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000198,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029493,"completed_at":"2026-11-07T11:05:07.314Z","load_balancing_strategy":"primary","db_duration_s":0.004124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.369968] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.316Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3b7f9fa293ab3a28daf5c89a","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3b7f9fa293ab3a28daf5c89a: start","job_status":"start","queue_duration_s":1.244767,"scheduling_latency_s":1.244799} vm-test-run-nixbot-gitlab> gitlab # [ 394.382780] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.331Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 394.384623] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.343Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3b7f9fa293ab3a28daf5c89a","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3b7f9fa293ab3a28daf5c89a: done: 0.027233 sec","job_status":"done","queue_duration_s":1.244767,"scheduling_latency_s":1.244799,"redis_calls":10,"redis_duration_s":0.002247,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027233,"completed_at":"2026-11-07T11:05:07.343Z","load_balancing_strategy":"primary","db_duration_s":0.002962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.424359] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.345Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87e3c5571d711a057fe2e358","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87e3c5571d711a057fe2e358: start","job_status":"start","queue_duration_s":1.273899,"scheduling_latency_s":1.27393} vm-test-run-nixbot-gitlab> gitlab # [ 394.438026] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.361Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 394.439844] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.372Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87e3c5571d711a057fe2e358","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87e3c5571d711a057fe2e358: done: 0.027077 sec","job_status":"done","queue_duration_s":1.273899,"scheduling_latency_s":1.27393,"redis_calls":10,"redis_duration_s":0.002235,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027077,"completed_at":"2026-11-07T11:05:07.372Z","load_balancing_strategy":"primary","db_duration_s":0.0022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 394.479941] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.374Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"387f1cf2fa6c5865c5a9ffed","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-387f1cf2fa6c5865c5a9ffed: start","job_status":"start","queue_duration_s":1.302818,"scheduling_latency_s":1.302893} vm-test-run-nixbot-gitlab> gitlab # [ 394.492849] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.388Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 394.494554] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:07.400Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"387f1cf2fa6c5865c5a9ffed","created_at":"2026-11-07T11:05:06.053Z","trace_propagation_headers":{"sentry-trace":"6319d7df4f3743cd84080347966ca22d-25f67f7707eb4a5e","baggage":"sentry-trace_id=6319d7df4f3743cd84080347966ca22d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2f45b85219d299bb25abf28cba8dd8d8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:05:06.071Z","job_size_bytes":2,"pid":2281,"sidekiq_tid":"m7px","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-387f1cf2fa6c5865c5a9ffed: done: 0.02573 sec","job_status":"done","queue_duration_s":1.302818,"scheduling_latency_s":1.302893,"redis_calls":10,"redis_duration_s":0.00208,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02573,"completed_at":"2026-11-07T11:05:07.400Z","load_balancing_strategy":"primary","db_duration_s":0.002085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 426.296511] postgres[4085]: [4085] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 426.296942] postgres[4085]: [4085] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 434.297739] runuser[3558]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 434.298862] setup-gitlab-start[3512]: + systemctl restart gitlab-sidekiq.service vm-test-run-nixbot-gitlab> gitlab # [ 434.342851] systemd[1]: Stopping gitlab-sidekiq.service... vm-test-run-nixbot-gitlab> gitlab # [ 434.344972] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:49.066Z","message":"Shutting down"} vm-test-run-nixbot-gitlab> gitlab # [ 434.347263] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:49.067Z","message":"Scheduler exiting..."} vm-test-run-nixbot-gitlab> gitlab # [ 434.349662] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:49.067Z","message":"Terminating quiet threads for default capsule"} vm-test-run-nixbot-gitlab> gitlab # [ 434.352511] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:49.067Z","message":"Scheduler exiting..."} vm-test-run-nixbot-gitlab> gitlab # [ 434.845104] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:49.568Z","message":"Pausing to allow jobs to finish..."} vm-test-run-nixbot-gitlab> gitlab # [ 438.349083] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:53.072Z","message":"Bye!"} vm-test-run-nixbot-gitlab> gitlab # [ 438.379452] sidekiq-cluster[2281]: {"severity":"INFO","time":"2026-11-07T11:05:53.103Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2281,"worker_id":"sidekiq_0","memwd_rss_bytes":936157184,"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 439.285768] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 439.286072] systemd[1]: Stopped gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 439.286202] systemd[1]: gitlab-sidekiq.service: Consumed 2min 43.626s CPU time over 4min 50.343s wall clock time, 937.9M memory peak, 4.5M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 439.294038] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 439.297178] setup-gitlab-start[3512]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 439.300398] setup-gitlab-start[3512]: + echo nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 439.301931] setup-gitlab-start[3512]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 439.304351] setup-gitlab-start[3512]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars vm-test-run-nixbot-gitlab> gitlab # [ 439.972806] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:05:54 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 634 vm-test-run-nixbot-gitlab> gitlab # [ 443.336224] gitaly[928]: time="2026-11-07T11:05:58.059Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.059" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.346823] gitaly[928]: time="2026-11-07T11:05:58.059Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.059" grpc.time_ms=0.625 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.599390] gitaly[928]: time="2026-11-07T11:05:58.323Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.272" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.322" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.616519] gitaly[928]: time="2026-11-07T11:05:58.323Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.272" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.322" grpc.time_ms=0.441 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.679927] gitaly[928]: time="2026-11-07T11:05:58.403Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW diskcache=1962032d-253b-4f85-97e3-7ad13590676f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-07T11:06:08.003" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.354" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.697243] gitaly[928]: time="2026-11-07T11:05:58.404Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-07T11:06:08.003" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.354" grpc.time_ms=49.903 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.719266] gitaly[928]: time="2026-11-07T11:05:58.424Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.073" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.423" pid=928 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.734276] gitaly[928]: time="2026-11-07T11:05:58.425Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.073" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.423" grpc.time_ms=0.998 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.748856] gitaly[928]: time="2026-11-07T11:05:58.448Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:08.084" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.435" grpc.time_ms=12.777 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.845778] gitaly[928]: time="2026-11-07T11:05:58.569Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.569" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.863146] gitaly[928]: time="2026-11-07T11:05:58.570Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.569" grpc.time_ms=0.656 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.897112] gitaly[928]: time="2026-11-07T11:05:58.620Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW diskcache=4472205b-756a-4cb2-844f-adc6d14c0685 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-07T11:06:08.224" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.574" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.910104] gitaly[928]: time="2026-11-07T11:05:58.621Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=28 command.spawn_token_fork_ms=1 command.system_time_ms=20 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-07T11:06:08.224" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.574" grpc.time_ms=46.878 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.926897] gitaly[928]: time="2026-11-07T11:05:58.635Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.284" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.634" pid=928 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.939993] gitaly[928]: time="2026-11-07T11:05:58.636Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.284" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.634" grpc.time_ms=0.941 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.952500] gitaly[928]: time="2026-11-07T11:05:58.655Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:08.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.643" grpc.time_ms=12.24 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 443.995427] gitaly[928]: time="2026-11-07T11:05:58.719Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:06:08.078" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.718" grpc.time_ms=0.368 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 444.014133] gitaly[928]: time="2026-11-07T11:05:58.732Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-07T11:06:08.091" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:58.731" grpc.time_ms=0.558 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 444.844958] gitaly[928]: time="2026-11-07T11:05:59.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZE8WQTD1TZNKZ7ZMC6PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:09.205" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:05:59.556" grpc.time_ms=11.993 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 445.292865] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:00 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 5299 vm-test-run-nixbot-gitlab> gitlab # [ 445.296067] setup-gitlab-start[4173]: {"id":1,"message":null,"error":null} vm-test-run-nixbot-gitlab> gitlab # [ 447.008295] sidekiq-cluster[4156]: {"severity":"INFO","time":"2026-11-07T11:06:01.685Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 451.018574] gitaly[928]: time="2026-11-07T11:06:05.742Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZRBPYQ5JM5DYK2532M39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:15.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:05.729" grpc.time_ms=11.9 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 451.720267] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:06 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1396 vm-test-run-nixbot-gitlab> gitlab # [ 451.725431] setup-gitlab-start[3512]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 451.726884] setup-gitlab-start[3512]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 451.728427] setup-gitlab-start[3512]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 451.770245] setup-gitlab-start[4270]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 451.772098] setup-gitlab-start[3512]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 451.775454] setup-gitlab-start[3512]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 451.780271] setup-gitlab-start[3512]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 451.784202] setup-gitlab-start[3512]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 451.790311] setup-gitlab-start[3512]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 451.835584] setup-gitlab-start[4275]: [master (root-commit) 4e706c3] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 451.837600] setup-gitlab-start[4275]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 451.839153] setup-gitlab-start[4275]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 451.840648] setup-gitlab-start[3512]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 451.845995] setup-gitlab-start[3512]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 451.946112] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 58 vm-test-run-nixbot-gitlab> gitlab # [ 452.321412] gitaly[928]: time="2026-11-07T11:06:07.044Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZSYFQMPP23DNPVDTE8CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:07.031" grpc.time_ms=12.734 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 452.343939] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 374 vm-test-run-nixbot-gitlab> gitlab # [ 452.680709] gitaly[928]: time="2026-11-07T11:06:07.404Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:17.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:07.391" grpc.time_ms=12.392 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 452.741559] gitaly[928]: time="2026-11-07T11:06:07.465Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-07T11:06:37.054" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-07T11:06:07.455" grpc.time_ms=9.973 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 452.789911] gitaly[928]: time="2026-11-07T11:06:07.513Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T duration_ms=258ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 452.857058] gitaly[928]: time="2026-11-07T11:06:07.581Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T duration_ms=61ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 452.864540] gitaly[928]: time="2026-11-07T11:06:07.581Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:07.253" grpc.time_ms=327.592 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 452.902071] gitaly[928]: time="2026-11-07T11:06:07.626Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:07.624" grpc.time_ms=1.054 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 452.957421] gitaly[928]: time="2026-11-07T11:06:07.681Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:07.679" grpc.time_ms=1.177 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.000972] gitaly[928]: time="2026-11-07T11:06:07.724Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:07.723" grpc.time_ms=1.772 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.178165] gitaly[928]: time="2026-11-07T11:06:07.902Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:06:17.249" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:07.889" grpc.time_ms=12.008 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.226994] gitaly[928]: time="2026-11-07T11:06:07.950Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1431 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=5 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-07T11:06:17.278" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:07.918" grpc.time_ms=31.676 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.279627] gitaly[928]: time="2026-11-07T11:06:08.003Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T duration_ms=222ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 453.284904] gitaly[928]: time="2026-11-07T11:06:08.004Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:07.779" grpc.time_ms=224.869 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.308975] gitaly[928]: time="2026-11-07T11:06:08.032Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T diskcache=0bfe67c4-d090-402d-85ae-df6569399ce2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:07.174" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.326297] gitaly[928]: time="2026-11-07T11:06:08.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=245 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13553 command.oublock=32 command.real_time_ms=849 command.spawn_token_fork_ms=0 command.system_time_ms=124 command.user_time_ms=121 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:07.174" grpc.time_ms=859.087 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 453.347729] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 453.349310] setup-gitlab-start[4279]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 453.350594] setup-gitlab-start[4279]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 453.351779] setup-gitlab-start[4279]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 453.353684] gitlab-workhorse[830]: time="2026-11-07T11:06:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 453.356103] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:08 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 974 vm-test-run-nixbot-gitlab> gitlab # [ 453.358501] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 454.98 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 455.504588] nixbot[4361]: {"ts": "2026-11-07T11:06:10.227Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 455.559615] nixbot[4361]: {"ts": "2026-11-07T11:06:10.283Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 455.589701] nixbot[4361]: {"ts": "2026-11-07T11:06:10.313Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 455.597314] nixbot[4361]: {"ts": "2026-11-07T11:06:10.321Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 455.604000] nixbot[4361]: {"ts": "2026-11-07T11:06:10.327Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 455.612653] nixbot[4361]: {"ts": "2026-11-07T11:06:10.336Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 455.619325] nixbot[4361]: {"ts": "2026-11-07T11:06:10.343Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 455.623051] nixbot[4361]: {"ts": "2026-11-07T11:06:10.345Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 455.631148] nixbot[4361]: {"ts": "2026-11-07T11:06:10.354Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 455.635086] nixbot[4361]: {"ts": "2026-11-07T11:06:10.358Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 455.642548] nixbot[4361]: {"ts": "2026-11-07T11:06:10.366Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 455.652487] nixbot[4361]: {"ts": "2026-11-07T11:06:10.376Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 455.656914] nixbot[4361]: {"ts": "2026-11-07T11:06:10.380Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 455.660642] nixbot[4361]: {"ts": "2026-11-07T11:06:10.383Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 455.666168] nixbot[4361]: {"ts": "2026-11-07T11:06:10.389Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 455.669959] nixbot[4361]: {"ts": "2026-11-07T11:06:10.393Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 455.673655] nixbot[4361]: {"ts": "2026-11-07T11:06:10.395Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 455.677153] nixbot[4361]: {"ts": "2026-11-07T11:06:10.397Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 455.681447] nixbot[4361]: {"ts": "2026-11-07T11:06:10.405Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} vm-test-run-nixbot-gitlab> gitlab # [ 455.684946] nixbot[4361]: {"ts": "2026-11-07T11:06:10.407Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 455.688470] nixbot[4361]: {"ts": "2026-11-07T11:06:10.409Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} vm-test-run-nixbot-gitlab> gitlab # [ 455.693623] nixbot[4361]: {"ts": "2026-11-07T11:06:10.417Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 455.697412] nixbot[4361]: {"ts": "2026-11-07T11:06:10.420Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} vm-test-run-nixbot-gitlab> gitlab # [ 455.700863] nixbot[4361]: {"ts": "2026-11-07T11:06:10.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} vm-test-run-nixbot-gitlab> gitlab # [ 455.704212] nixbot[4361]: {"ts": "2026-11-07T11:06:10.426Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 455.707691] nixbot[4361]: {"ts": "2026-11-07T11:06:10.430Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} vm-test-run-nixbot-gitlab> gitlab # [ 455.711808] nixbot[4361]: {"ts": "2026-11-07T11:06:10.433Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 455.715250] nixbot[4361]: {"ts": "2026-11-07T11:06:10.436Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} vm-test-run-nixbot-gitlab> gitlab # [ 455.719257] nixbot[4361]: {"ts": "2026-11-07T11:06:10.438Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 455.732519] nixbot[4361]: {"ts": "2026-11-07T11:06:10.456Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 455.740330] nixbot[4361]: {"ts": "2026-11-07T11:06:10.464Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} vm-test-run-nixbot-gitlab> gitlab # [ 455.743845] nixbot[4361]: {"ts": "2026-11-07T11:06:10.466Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} vm-test-run-nixbot-gitlab> gitlab # [ 455.747460] nixbot[4361]: {"ts": "2026-11-07T11:06:10.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 455.751542] nixbot[4361]: {"ts": "2026-11-07T11:06:10.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} vm-test-run-nixbot-gitlab> gitlab # [ 455.755112] nixbot[4361]: {"ts": "2026-11-07T11:06:10.477Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} vm-test-run-nixbot-gitlab> gitlab # [ 455.762145] nixbot[4361]: {"ts": "2026-11-07T11:06:10.486Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 455.765948] nixbot[4361]: {"ts": "2026-11-07T11:06:10.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} vm-test-run-nixbot-gitlab> gitlab # [ 455.771119] nixbot[4361]: {"ts": "2026-11-07T11:06:10.495Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"} vm-test-run-nixbot-gitlab> gitlab # [ 455.774887] nixbot[4361]: {"ts": "2026-11-07T11:06:10.498Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"} vm-test-run-nixbot-gitlab> gitlab # [ 456.323123] nixbot[4361]: {"ts": "2026-11-07T11:06:11.046Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [4361]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"} vm-test-run-nixbot-gitlab> gitlab # [ 456.327225] nixbot[4361]: {"ts": "2026-11-07T11:06:11.048Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"} vm-test-run-nixbot-gitlab> gitlab # [ 456.366907] nixbot[4361]: {"ts": "2026-11-07T11:06:11.090Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:36696 - \"GET /health HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 456.377227] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 456.379978] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 456.381758] systemd[1]: Startup finished in 885ms (kernel) + 5.183s (initrd) + 7min 30.313s (userspace) = 7min 36.381s. vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 456.627610] nixbot[4361]: {"ts": "2026-11-07T11:06:11.349Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:36698 - \"GET /health HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 456.770363] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 98 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 456.896704] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:11 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 530 vm-test-run-nixbot-gitlab> gitlab # [ 456.922561] nixbot[4361]: {"ts": "2026-11-07T11:06:11.646Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 457.038109] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 99 vm-test-run-nixbot-gitlab> gitlab # [ 457.042397] nixbot[4361]: {"ts": "2026-11-07T11:06:11.766Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab # [ 457.257128] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 211 vm-test-run-nixbot-gitlab> gitlab # [ 457.480174] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 210 vm-test-run-nixbot-gitlab> gitlab # [ 457.568760] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 73 vm-test-run-nixbot-gitlab> gitlab # [ 457.663660] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 88 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 457.777322] gitaly[928]: time="2026-11-07T11:06:12.500Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZZHD5N640M6BAGWF1RKM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-07T11:06:42.082" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:12.483" grpc.time_ms=16.541 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 457.813859] gitaly[928]: time="2026-11-07T11:06:12.537Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZZHD5N640M6BAGWF1RKM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:42.135" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:12.536" grpc.time_ms=0.913 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 457.852126] gitaly[928]: time="2026-11-07T11:06:12.558Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZZHD5N640M6BAGWF1RKM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-07T11:06:22.193" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:12.543" grpc.time_ms=14.441 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 457.900450] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 222 vm-test-run-nixbot-gitlab> gitlab # [ 457.943383] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 108 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.32 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' vm-test-run-nixbot-gitlab> gitlab # [ 458.045877] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 143 vm-test-run-nixbot-gitlab> gitlab # [ 458.053858] nixbot[4361]: {"ts": "2026-11-07T11:06:12.777Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 458.246252] gitaly[928]: time="2026-11-07T11:06:12.970Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV00039K9EE67TN5B9H3FZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:12.955" grpc.time_ms=14.27 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.268122] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 102 vm-test-run-nixbot-gitlab> gitlab # [ 458.449977] gitaly[928]: time="2026-11-07T11:06:13.173Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV003CDDD2XY9PNKBKAYQP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.160" pid=928 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.467484] gitaly[928]: time="2026-11-07T11:06:13.174Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV003CDDD2XY9PNKBKAYQP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.160" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.488174] gitlab-workhorse[830]: time="2026-11-07T11:06:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 vm-test-run-nixbot-gitlab> gitlab # [ 458.490731] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 202 vm-test-run-nixbot-gitlab> gitlab # [ 458.493346] gitaly[928]: time="2026-11-07T11:06:13.174Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV003CDDD2XY9PNKBKAYQP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.160" grpc.time_ms=14.043 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.588567] gitaly[928]: time="2026-11-07T11:06:13.312Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV009XM556DE9P296AVGPM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.287" pid=928 remote_ip="2001:db8:1::1" request_sha=a7aa86ee7a74161345f1b37bf2f65be3af695abb response_bytes=223 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.603625] gitaly[928]: time="2026-11-07T11:06:13.312Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV009XM556DE9P296AVGPM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.287" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.621729] gitlab-workhorse[830]: time="2026-11-07T11:06:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 vm-test-run-nixbot-gitlab> gitlab # [ 458.624514] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 131 vm-test-run-nixbot-gitlab> gitlab # [ 458.627246] gitaly[928]: time="2026-11-07T11:06:13.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV009XM556DE9P296AVGPM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.287" grpc.time_ms=24.935 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.747620] gitaly[928]: time="2026-11-07T11:06:13.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV00FFBEMZKYHWV7DEEJ03 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.460" grpc.time_ms=10.681 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.763302] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 113 vm-test-run-nixbot-gitlab> gitlab # [ 458.882113] gitaly[928]: time="2026-11-07T11:06:13.605Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00K5JZX9MX17TDMX45DZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.591" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.899626] gitaly[928]: time="2026-11-07T11:06:13.605Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00K5JZX9MX17TDMX45DZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.591" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 458.921931] gitlab-workhorse[830]: time="2026-11-07T11:06:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 458.924796] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 129 vm-test-run-nixbot-gitlab> gitlab # [ 458.927528] gitaly[928]: time="2026-11-07T11:06:13.605Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00K5JZX9MX17TDMX45DZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.591" grpc.time_ms=13.917 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.050052] gitaly[928]: time="2026-11-07T11:06:13.773Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV00S83YMQWJEH371H86TY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.760" grpc.time_ms=13.117 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.071869] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 102 vm-test-run-nixbot-gitlab> gitlab # [ 459.169362] gitaly[928]: time="2026-11-07T11:06:13.893Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00WHWHR85V2H034381AV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.869" pid=928 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.187284] gitaly[928]: time="2026-11-07T11:06:13.893Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1100 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00WHWHR85V2H034381AV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.869" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.210202] gitlab-workhorse[830]: time="2026-11-07T11:06:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 vm-test-run-nixbot-gitlab> gitlab # [ 459.213554] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 116 vm-test-run-nixbot-gitlab> gitlab # [ 459.216582] gitaly[928]: time="2026-11-07T11:06:13.893Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1100 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV00WHWHR85V2H034381AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:13.869" grpc.time_ms=23.776 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.312433] gitaly[928]: time="2026-11-07T11:06:14.036Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:44.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:14.035" grpc.time_ms=0.752 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.392417] gitaly[928]: time="2026-11-07T11:06:14.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-07T11:06:44.103" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:14.103" grpc.time_ms=12.702 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.430527] gitaly[928]: time="2026-11-07T11:06:14.154Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:06:24.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:14.142" grpc.time_ms=11.96 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.459640] gitaly[928]: time="2026-11-07T11:06:14.183Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:06:24.029" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:14.169" grpc.time_ms=13.799 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.602311] gitaly[928]: time="2026-11-07T11:06:14.326Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-07T11:06:24.185" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:14.325" grpc.time_ms=0.46 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.649512] gitaly[928]: time="2026-11-07T11:06:14.373Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-07T11:06:24.219" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:14.360" grpc.time_ms=12.599 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.732841] gitaly[928]: time="2026-11-07T11:06:14.456Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:14.455" grpc.time_ms=0.971 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.775548] gitaly[928]: time="2026-11-07T11:06:14.498Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:06:14.496" grpc.time_ms=2.431 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 459.792264] gitaly[928]: time="2026-11-07T11:06:14.508Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS diskcache=dadc1dd0-ed2d-4b06-a6f0-0183d79ccf71 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:06:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:14.399" pid=928 ref=refs/keep-around/4e706c3ecda168d9d017ec091fafdbae76772037 remote_ip="2001:db8:1::1" revision=4e706c3ecda168d9d017ec091fafdbae76772037 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.810016] gitaly[928]: time="2026-11-07T11:06:14.509Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5602 command.oublock=8 command.real_time_ms=101 command.spawn_token_fork_ms=1 command.system_time_ms=58 command.user_time_ms=48 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:06:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:06:14.399" grpc.time_ms=108.744 pid=928 ref=refs/keep-around/4e706c3ecda168d9d017ec091fafdbae76772037 remote_ip="2001:db8:1::1" revision=4e706c3ecda168d9d017ec091fafdbae76772037 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 459.830412] gitaly[928]: time="2026-11-07T11:06:14.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:44.115" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:14.516" grpc.time_ms=0.583 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 460.036767] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 819 vm-test-run-nixbot-gitlab> gitlab # [ 460.064228] nixbot[4361]: {"ts": "2026-11-07T11:06:14.787Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4e706c3ecda1-0"} vm-test-run-nixbot-gitlab> gitlab # [ 460.352485] nixbot[4361]: {"ts": "2026-11-07T11:06:15.076Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4e706c3ecda1-0", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 460.501205] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 460.593886] nix-daemon[4531]: accepted connection from pid 4526, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 465.035417] nix-daemon[4531]: accepted connection from pid 4570, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 465.052464] nix-daemon[4531]: accepted connection from pid 4526, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 469.904120] nix-daemon[4531]: accepted connection from pid 4526, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 470.074975] gitaly[928]: time="2026-11-07T11:06:24.798Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0BHGEPEM4ZXQDPT1XRJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:54.390" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:24.791" grpc.time_ms=6.688 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 470.568949] gitaly[928]: time="2026-11-07T11:06:25.292Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M6TV0BHGEPEM4ZXQDPT1XRJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:06:35.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:25.280" grpc.time_ms=12.548 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 470.777313] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 811 vm-test-run-nixbot-gitlab> gitlab # [ 470.870958] gitaly[928]: time="2026-11-07T11:06:25.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0CB73M9QV8DNSSEX7VFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:55.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:25.593" grpc.time_ms=0.967 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 470.937343] gitaly[928]: time="2026-11-07T11:06:25.661Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV0CB73M9QV8DNSSEX7VFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:06:35.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:25.650" grpc.time_ms=11.032 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 471.059547] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 271 vm-test-run-nixbot-gitlab> gitlab # [ 471.185652] gitaly[928]: time="2026-11-07T11:06:25.909Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0CMARSEM3C3H8MH66XSW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:06:55.507" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:06:25.907" grpc.time_ms=0.903 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 471.254457] gitaly[928]: time="2026-11-07T11:06:25.978Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV0CMARSEM3C3H8MH66XSW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:06:35.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:06:25.965" grpc.time_ms=12.209 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 471.356204] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 277 vm-test-run-nixbot-gitlab> gitlab # [ 471.689156] nixbot[4361]: {"ts": "2026-11-07T11:06:26.412Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "4e706c3ecda168d9d017ec091fafdbae76772037", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} vm-test-run-nixbot-gitlab> gitlab # [ 471.836418] gitaly[928]: time="2026-11-07T11:06:26.560Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV0D94SE2F12ES0FT28RRM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.548" grpc.time_ms=12.143 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 471.855537] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 93 vm-test-run-nixbot-gitlab> gitlab # [ 471.957056] gitaly[928]: time="2026-11-07T11:06:26.680Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DC43F5PT0PC9CCTC3AG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.667" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 471.973952] gitaly[928]: time="2026-11-07T11:06:26.681Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DC43F5PT0PC9CCTC3AG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.667" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 471.995421] gitlab-workhorse[830]: time="2026-11-07T11:06:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 471.998318] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 118 vm-test-run-nixbot-gitlab> gitlab # [ 472.001041] gitaly[928]: time="2026-11-07T11:06:26.681Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DC43F5PT0PC9CCTC3AG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.667" grpc.time_ms=14.138 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 472.131194] gitaly[928]: time="2026-11-07T11:06:26.854Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV0DJ76ZQR32K50CMDK7TE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.842" grpc.time_ms=11.865 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 472.145945] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 95 vm-test-run-nixbot-gitlab> gitlab # [ 472.247970] gitaly[928]: time="2026-11-07T11:06:26.971Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DN9X7YRBSKT0S8FY5EV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.957" pid=928 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 472.264840] gitaly[928]: time="2026-11-07T11:06:26.971Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DN9X7YRBSKT0S8FY5EV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.957" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 472.287146] gitlab-workhorse[830]: time="2026-11-07T11:06:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 472.290313] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:06:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 115 vm-test-run-nixbot-gitlab> gitlab # [ 472.293388] gitaly[928]: time="2026-11-07T11:06:26.971Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV0DN9X7YRBSKT0S8FY5EV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:06:26.957" grpc.time_ms=13.837 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 506.117849] sidekiq-cluster[4230]: 2026-11-07T11:07:00.840Z pid=4230 tid=z72 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>4230, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425213952} vm-test-run-nixbot-gitlab> gitlab # [ 506.535351] sidekiq-cluster[4230]: W, [2026-11-07T11:07:01.259223 #4230] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 506.712542] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:01.436Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 510.333536] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:05.056Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 510.333830] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:05.057Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 554.217284] postgres[5278]: [5278] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 554.217730] postgres[5278]: [5278] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 559.720534] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:54.436Z","message":"Booted Rails 7.2.3.1 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 559.720839] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:54.438Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 559.720910] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:54.438Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 559.720974] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:54.438Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 559.721035] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:54.440Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 101.91 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 560.038279] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:07:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 105 vm-test-run-nixbot-gitlab> gitlab # [ 560.192293] gitaly[928]: time="2026-11-07T11:07:54.915Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33GA6H05AQY1K35D8TJG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:07:54.902" grpc.time_ms=13.278 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # [ 560.214599] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:07:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 153 vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 560.559605] gitaly[928]: time="2026-11-07T11:07:55.283Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-07T11:08:25.271" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-07T11:07:55.271" grpc.time_ms=11.451 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 560.589245] gitaly[928]: time="2026-11-07T11:07:55.313Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M6TV33P8EXV4540VK1ZFAPDF duration_ms=151ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 560.638468] gitaly[928]: time="2026-11-07T11:07:55.362Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M6TV33P8EXV4540VK1ZFAPDF duration_ms=48ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 560.644165] gitaly[928]: time="2026-11-07T11:07:55.362Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:55.159" grpc.time_ms=203.071 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 560.686234] gitaly[928]: time="2026-11-07T11:07:55.410Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:55.409" grpc.time_ms=1.077 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 560.743308] gitaly[928]: time="2026-11-07T11:07:55.467Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:55.465" grpc.time_ms=1.525 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 560.782594] gitaly[928]: time="2026-11-07T11:07:55.506Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:55.505" grpc.time_ms=0.781 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 560.940471] gitaly[928]: time="2026-11-07T11:07:55.664Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M6TV33P8EXV4540VK1ZFAPDF duration_ms=120ns method=POST pid=928 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 560.946188] gitaly[928]: time="2026-11-07T11:07:55.665Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:55.543" grpc.time_ms=121.759 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 560.965033] gitaly[928]: time="2026-11-07T11:07:55.676Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF diskcache=45e15fe4-f1b0-4115-a3be-62e458cdd352 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:07:55.085" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # 4e706c3..df7e7ae [ 560.981708] gitlab-workhorse[830]: time="2026-11-07T11:07:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # master -> master[ 560.984977] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:07:55 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 724 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.13 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab # [ 560.988273] gitaly[928]: time="2026-11-07T11:07:55.676Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=247 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13379 command.oublock=32 command.real_time_ms=584 command.spawn_token_fork_ms=0 command.system_time_ms=130 command.user_time_ms=117 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:07:55.085" grpc.time_ms=590.749 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 561.124007] nixbot[4361]: {"ts": "2026-11-07T11:07:55.847Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43156 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.11 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 562.163291] nixbot[4361]: {"ts": "2026-11-07T11:07:56.886Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43162 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 562.974588] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:57.697Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"dd150b49610aab54f72d4e0b","created_at":"2026-11-07T11:05:58.466Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:58.527Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-dd150b49610aab54f72d4e0b: start","job_status":"start","queue_duration_s":119.169499,"scheduling_latency_s":119.169565} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 563.196963] nixbot[4361]: {"ts": "2026-11-07T11:07:57.919Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43176 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.03 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 563.232060] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:57.955Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"dd150b49610aab54f72d4e0b","created_at":"2026-11-07T11:05:58.466Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:58.527Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-dd150b49610aab54f72d4e0b: done: 0.258041 sec","job_status":"done","queue_duration_s":119.169499,"scheduling_latency_s":119.169565,"redis_calls":23,"redis_duration_s":0.007448,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003902,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001954,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001592,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1322478,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.258041,"completed_at":"2026-11-07T11:07:57.954Z","load_balancing_strategy":"primary","db_duration_s":0.077371,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 563.301040] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:57.958Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3a918e9621d695f4429cb940","created_at":"2026-11-07T11:05:58.548Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:58.560Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3a918e9621d695f4429cb940: start","job_status":"start","queue_duration_s":119.397006,"scheduling_latency_s":119.397049} vm-test-run-nixbot-gitlab> gitlab # [ 563.392614] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.115Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3a918e9621d695f4429cb940","created_at":"2026-11-07T11:05:58.548Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:58.560Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3a918e9621d695f4429cb940: done: 0.157966 sec","job_status":"done","queue_duration_s":119.397006,"scheduling_latency_s":119.397049,"redis_calls":14,"redis_duration_s":0.004956,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001064,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00218,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11456924,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.157966,"completed_at":"2026-11-07T11:07:58.115Z","load_balancing_strategy":"primary","db_duration_s":0.029349,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 563.464174] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.118Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ea5dea3c2af5722687644cbe","created_at":"2026-11-07T11:05:59.373Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.387Z","job_size_bytes":5,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ea5dea3c2af5722687644cbe: start","job_status":"start","queue_duration_s":118.731011,"scheduling_latency_s":118.731049} vm-test-run-nixbot-gitlab> gitlab # [ 563.623560] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.346Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ea5dea3c2af5722687644cbe","created_at":"2026-11-07T11:05:59.373Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.387Z","job_size_bytes":5,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ea5dea3c2af5722687644cbe: done: 0.228146 sec","job_status":"done","queue_duration_s":118.731011,"scheduling_latency_s":118.731049,"redis_calls":12,"redis_duration_s":0.003261,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000868,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001874,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.18666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045062,"duration_s":0.228146,"completed_at":"2026-11-07T11:07:58.346Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 563.698131] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.349Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"084d422d3607567b8dfd256a","created_at":"2026-11-07T11:05:59.443Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.455Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-084d422d3607567b8dfd256a: start","job_status":"start","queue_duration_s":118.893602,"scheduling_latency_s":118.893636} vm-test-run-nixbot-gitlab> gitlab # [ 563.967316] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.690Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"084d422d3607567b8dfd256a","created_at":"2026-11-07T11:05:59.443Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.455Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-084d422d3607567b8dfd256a: done: 0.341398 sec","job_status":"done","queue_duration_s":118.893602,"scheduling_latency_s":118.893636,"redis_calls":11,"redis_duration_s":0.003284,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000785,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":36,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.283,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.283,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.247789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000689,"duration_s":0.341398,"completed_at":"2026-11-07T11:07:58.690Z","load_balancing_strategy":"primary","db_duration_s":0.100543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 564.036084] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.693Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"6747488bf8251b9fc661b75f","created_at":"2026-11-07T11:05:59.460Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.470Z","job_size_bytes":89,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-6747488bf8251b9fc661b75f: start","job_status":"start","queue_duration_s":119.223478,"scheduling_latency_s":119.223521} vm-test-run-nixbot-gitlab> gitlab # [ 564.056264] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.756Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"6747488bf8251b9fc661b75f","created_at":"2026-11-07T11:05:59.460Z","trace_propagation_headers":{"sentry-trace":"e6e7844b746947c2a4f04c7fba6f38db-6e121563b4ea46a0","baggage":"sentry-trace_id=e6e7844b746947c2a4f04c7fba6f38db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZE8WQTD1TZNKZ7ZMC6PW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-07T11:05:59.470Z","job_size_bytes":89,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-6747488bf8251b9fc661b75f: done: 0.063388 sec","job_status":"done","queue_duration_s":119.223478,"scheduling_latency_s":119.223521,"redis_calls":11,"redis_duration_s":0.002905,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000826,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041402,"duration_s":0.063388,"completed_at":"2026-11-07T11:07:58.756Z","load_balancing_strategy":"primary","db_duration_s":0.00793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 564.105477] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.759Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4e706c3ecda168d9d017ec091fafdbae76772037 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"34c6b52b49bf94fc56694f84","created_at":"2026-11-07T11:06:07.849Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-78c6cda7bb984721","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:06:07.858Z","job_size_bytes":185,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-34c6b52b49bf94fc56694f84: start","job_status":"start","queue_duration_s":110.900879,"scheduling_latency_s":110.900912} vm-test-run-nixbot-gitlab> gitlab # [ 564.123881] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:07:58.786Z","message":"changes: \"0000000000000000000000000000000000000000 4e706c3ecda168d9d017ec091fafdbae76772037 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 564.126904] gitaly[928]: time="2026-11-07T11:07:58.838Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:08.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:07:58.826" grpc.time_ms=12.469 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.140913] gitaly[928]: time="2026-11-07T11:07:58.858Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:08:08.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:07:58.846" grpc.time_ms=11.8 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.05 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 564.238756] gitaly[928]: time="2026-11-07T11:07:58.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-07T11:08:08.126" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:07:58.946" grpc.time_ms=15.575 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.259592] nixbot[4361]: {"ts": "2026-11-07T11:07:58.972Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43192 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 564.299342] gitaly[928]: time="2026-11-07T11:07:59.023Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:59.022" grpc.time_ms=0.71 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 564.346092] gitaly[928]: time="2026-11-07T11:07:59.070Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:07:59.069" grpc.time_ms=0.64 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 564.362397] gitaly[928]: time="2026-11-07T11:07:59.081Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T diskcache=d7ca00de-2ec2-417f-94c6-37886e44a850 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:08:08.162" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:07:58.983" pid=928 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.378506] gitaly[928]: time="2026-11-07T11:07:59.082Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=97 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5163 command.oublock=8 command.real_time_ms=91 command.spawn_token_fork_ms=1 command.system_time_ms=51 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:08:08.162" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:07:58.983" grpc.time_ms=98.667 pid=928 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.396453] gitaly[928]: time="2026-11-07T11:07:59.105Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-07T11:08:09.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:07:59.105" grpc.time_ms=0.462 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.409038] gitaly[928]: time="2026-11-07T11:07:59.131Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-07T11:08:09.012" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:07:59.113" grpc.time_ms=18.074 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.560460] gitaly[928]: time="2026-11-07T11:07:59.284Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-07T11:08:29.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:07:59.275" grpc.time_ms=8.677 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.607285] gitaly[928]: time="2026-11-07T11:07:59.331Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-07T11:08:09.230" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:07:59.330" grpc.time_ms=0.448 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.624656] gitaly[928]: time="2026-11-07T11:07:59.342Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-07T11:08:09.234" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-07T11:07:59.334" grpc.time_ms=8.065 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.658101] gitaly[928]: time="2026-11-07T11:07:59.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:09.268" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:07:59.369" grpc.time_ms=12.611 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.680600] gitaly[928]: time="2026-11-07T11:07:59.404Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:09.291" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:07:59.391" grpc.time_ms=12.687 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 564.723307] gitaly[928]: time="2026-11-07T11:07:59.447Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-07T11:08:29.226" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:07:59.426" grpc.time_ms=20.749 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 565.296081] nixbot[4361]: {"ts": "2026-11-07T11:08:00.019Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43208 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 565.367834] gitaly[928]: time="2026-11-07T11:08:00.091Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-07T11:08:10.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:00.088" grpc.time_ms=2.057 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 565.422053] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.144Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4e706c3ecda168d9d017ec091fafdbae76772037 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"34c6b52b49bf94fc56694f84","created_at":"2026-11-07T11:06:07.849Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-78c6cda7bb984721","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:06:07.858Z","job_size_bytes":185,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-34c6b52b49bf94fc56694f84: done: 1.384645 sec","job_status":"done","queue_duration_s":110.900879,"scheduling_latency_s":110.900912,"gitaly_calls":13,"gitaly_duration_s":0.276757,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.06131,"redis_read_bytes":2103,"redis_write_bytes":21746,"redis_cache_calls":15,"redis_cache_duration_s":0.005648,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008536,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002475,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7834,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007116,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022863,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014672,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.327,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.369,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.98549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00119538,"exclusive_lock_hold_duration_s":0.03836744,"duration_s":1.384645,"completed_at":"2026-11-07T11:08:00.144Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.136468,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 565.499508] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.147Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f1f9489e878730713a62cb4f","created_at":"2026-11-07T11:06:14.529Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.542Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f1f9489e878730713a62cb4f: start","job_status":"start","queue_duration_s":105.604608,"scheduling_latency_s":105.60477} vm-test-run-nixbot-gitlab> gitlab # [ 565.578266] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.300Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f1f9489e878730713a62cb4f","created_at":"2026-11-07T11:06:14.529Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.542Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f1f9489e878730713a62cb4f: done: 0.153379 sec","job_status":"done","queue_duration_s":105.604608,"scheduling_latency_s":105.60477,"redis_calls":16,"redis_duration_s":0.005893,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003045,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048218,"duration_s":0.153379,"completed_at":"2026-11-07T11:08:00.300Z","load_balancing_strategy":"primary","db_duration_s":0.028709,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 565.643377] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.304Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c11e06c2f6fe0b12937dd5f6","created_at":"2026-11-07T11:06:14.543Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.553Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c11e06c2f6fe0b12937dd5f6: start","job_status":"start","queue_duration_s":105.751183,"scheduling_latency_s":105.751233} vm-test-run-nixbot-gitlab> gitlab # [ 565.762093] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.485Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c11e06c2f6fe0b12937dd5f6","created_at":"2026-11-07T11:06:14.543Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.553Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c11e06c2f6fe0b12937dd5f6: done: 0.180345 sec","job_status":"done","queue_duration_s":105.751183,"scheduling_latency_s":105.751233,"redis_calls":21,"redis_duration_s":0.007597,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003234,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002141,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078058,"duration_s":0.180345,"completed_at":"2026-11-07T11:08:00.484Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 565.828569] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:00.488Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bdd8e3cae7985d46cd0fb123","created_at":"2026-11-07T11:06:14.722Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.732Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bdd8e3cae7985d46cd0fb123: start","job_status":"start","queue_duration_s":105.755431,"scheduling_latency_s":105.755472} vm-test-run-nixbot-gitlab> gitlab # [ 566.236237] gitaly[928]: time="2026-11-07T11:08:00.960Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:00.959" grpc.time_ms=0.621 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 566.274750] gitaly[928]: time="2026-11-07T11:08:00.998Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:00.998" grpc.time_ms=0.608 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 566.328482] gitaly[928]: time="2026-11-07T11:08:01.052Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:01.051" grpc.time_ms=0.648 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 566.346167] nixbot[4361]: {"ts": "2026-11-07T11:08:01.062Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43214 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 566.356923] gitaly[928]: time="2026-11-07T11:08:01.080Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS diskcache=fc8460aa-2d39-41e0-a7c3-d274fd1f7f6c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:30.719" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:00.920" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.373373] gitaly[928]: time="2026-11-07T11:08:01.081Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=152 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7562 command.oublock=0 command.real_time_ms=153 command.spawn_token_fork_ms=0 command.system_time_ms=106 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:30.719" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:00.920" grpc.time_ms=160.501 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.541558] gitaly[928]: time="2026-11-07T11:08:01.265Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:31.063" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.263" grpc.time_ms=1.346 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.608981] gitaly[928]: time="2026-11-07T11:08:01.332Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV011KN200XPJ4BA8PBQPS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:31.130" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.331" grpc.time_ms=1.207 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.645519] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.368Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bdd8e3cae7985d46cd0fb123","created_at":"2026-11-07T11:06:14.722Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-877347345e6d40e9","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:06:14.732Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bdd8e3cae7985d46cd0fb123: done: 0.879963 sec","job_status":"done","queue_duration_s":105.755431,"scheduling_latency_s":105.755472,"gitaly_calls":3,"gitaly_duration_s":0.173244,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.036079,"redis_read_bytes":455,"redis_write_bytes":28003,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002584,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002008,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012426,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.004666,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14497,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007198,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000777,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000693,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003143,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.081,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.04,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.042,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.606413,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.76597066,"duration_s":0.879963,"completed_at":"2026-11-07T11:08:01.367Z","load_balancing_strategy":"primary","db_duration_s":0.095951,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 566.721345] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.370Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4e706c3ecda168d9d017ec091fafdbae76772037 df7e7ae3022955d8a907d68546822b663a3d4ff9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a9326e317eb5ec40e9fd783c","created_at":"2026-11-07T11:07:55.606Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-662e28e517cb4515","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:07:55.613Z","job_size_bytes":185,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a9326e317eb5ec40e9fd783c: start","job_status":"start","queue_duration_s":5.757123,"scheduling_latency_s":5.757159} vm-test-run-nixbot-gitlab> gitlab # [ 566.740401] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.390Z","message":"changes: \"4e706c3ecda168d9d017ec091fafdbae76772037 df7e7ae3022955d8a907d68546822b663a3d4ff9 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 566.743045] gitaly[928]: time="2026-11-07T11:08:01.429Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:11.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:01.417" grpc.time_ms=12.53 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.757682] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.459Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"44ed0d63bea454757fb22897","created_at":"2026-11-07T11:08:01.456Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"2d0c8189be50a89c65dfa1eb","job_size_bytes":46,"pid":4230,"job_status":"deduplicated","message":"ProjectCacheWorker JID-44ed0d63bea454757fb22897: deduplicated: until executing","deduplication.type":"until executing"} vm-test-run-nixbot-gitlab> gitlab # [ 566.774063] gitaly[928]: time="2026-11-07T11:08:01.450Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-07T11:08:11.028" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:08:01.438" grpc.time_ms=11.85 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.788291] gitaly[928]: time="2026-11-07T11:08:01.497Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-07T11:08:11.073" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:01.483" grpc.time_ms=13.261 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.802011] gitaly[928]: time="2026-11-07T11:08:01.520Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-07T11:08:31.306" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.506" grpc.time_ms=13.79 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.823303] gitaly[928]: time="2026-11-07T11:08:01.547Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-07T11:08:11.128" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-07T11:08:01.539" grpc.time_ms=7.777 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 566.852232] gitaly[928]: time="2026-11-07T11:08:01.576Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:11.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:01.563" grpc.time_ms=12.406 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.007000] gitaly[928]: time="2026-11-07T11:08:01.730Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-07T11:08:11.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.728" grpc.time_ms=2.191 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.047014] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.770Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4e706c3ecda168d9d017ec091fafdbae76772037 df7e7ae3022955d8a907d68546822b663a3d4ff9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a9326e317eb5ec40e9fd783c","created_at":"2026-11-07T11:07:55.606Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-662e28e517cb4515","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:07:55.613Z","job_size_bytes":185,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a9326e317eb5ec40e9fd783c: done: 0.399641 sec","job_status":"done","queue_duration_s":5.757123,"scheduling_latency_s":5.757159,"gitaly_calls":7,"gitaly_duration_s":0.101668,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.030788,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.002973,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001513,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00569,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008993,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.011619,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.036,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.036,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.266057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001469,"exclusive_lock_hold_duration_s":0.02651142,"duration_s":0.399641,"completed_at":"2026-11-07T11:08:01.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035943,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 567.125183] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.772Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2d0c8189be50a89c65dfa1eb","created_at":"2026-11-07T11:07:58.863Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:58.889Z","job_size_bytes":46,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2d0c8189be50a89c65dfa1eb: start","job_status":"start","queue_duration_s":2.883637,"scheduling_latency_s":2.883675} vm-test-run-nixbot-gitlab> gitlab # [ 567.144026] gitaly[928]: time="2026-11-07T11:08:01.855Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-07T17:08:01.848" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:08:01.848" grpc.time_ms=6.131 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.157175] gitaly[928]: time="2026-11-07T11:08:01.879Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-07T11:08:31.664" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.865" grpc.time_ms=13.297 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.203330] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.926Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2d0c8189be50a89c65dfa1eb","created_at":"2026-11-07T11:07:58.863Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:58.889Z","job_size_bytes":46,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2d0c8189be50a89c65dfa1eb: done: 0.153766 sec","job_status":"done","queue_duration_s":2.883637,"scheduling_latency_s":2.883675,"gitaly_calls":2,"gitaly_duration_s":0.025724,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008913,"redis_read_bytes":69,"redis_write_bytes":4881,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001864,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000785,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001994,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003358,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004446,"duration_s":0.153766,"completed_at":"2026-11-07T11:08:01.926Z","load_balancing_strategy":"primary","db_duration_s":0.016937,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 567.265004] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:01.929Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"60f9b823a38c5b8b02ffb481","created_at":"2026-11-07T11:07:58.930Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:58.940Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-60f9b823a38c5b8b02ffb481: start","job_status":"start","queue_duration_s":2.988953,"scheduling_latency_s":2.988992} vm-test-run-nixbot-gitlab> gitlab # [ 567.283666] gitaly[928]: time="2026-11-07T11:08:02.004Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T diskcache=7db1ae2b-d984-4bda-a4eb-5f3675099549 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-07T17:08:01.970" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.971" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.295429] gitaly[928]: time="2026-11-07T11:08:02.005Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=37 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-07T17:08:01.970" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:01.971" grpc.time_ms=33.132 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.05 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 567.351852] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:02.075Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"60f9b823a38c5b8b02ffb481","created_at":"2026-11-07T11:07:58.930Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:58.940Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-60f9b823a38c5b8b02ffb481: done: 0.146014 sec","job_status":"done","queue_duration_s":2.988953,"scheduling_latency_s":2.988992,"gitaly_calls":1,"gitaly_duration_s":0.03719,"redis_calls":15,"redis_duration_s":0.004539,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000563,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000637,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001599,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.018,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.035,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1029828,"duration_s":0.146014,"completed_at":"2026-11-07T11:08:02.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 567.417417] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:02.078Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8373d30cc0614972e88f82d5","created_at":"2026-11-07T11:07:59.096Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:59.100Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8373d30cc0614972e88f82d5: start","job_status":"start","queue_duration_s":2.977494,"scheduling_latency_s":2.977565} vm-test-run-nixbot-gitlab> gitlab # [ 567.435876] nixbot[4361]: {"ts": "2026-11-07T11:08:02.109Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43230 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 567.439488] gitaly[928]: time="2026-11-07T11:08:02.148Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-07T11:08:32.035" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:02.135" grpc.time_ms=12.554 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.453326] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:02.167Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8373d30cc0614972e88f82d5","created_at":"2026-11-07T11:07:59.096Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:59.100Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8373d30cc0614972e88f82d5: done: 0.089071 sec","job_status":"done","queue_duration_s":2.977494,"scheduling_latency_s":2.977565,"gitaly_calls":1,"gitaly_duration_s":0.016243,"redis_calls":16,"redis_duration_s":0.006687,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000495,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003374,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089071,"completed_at":"2026-11-07T11:08:02.167Z","load_balancing_strategy":"primary","db_duration_s":0.007309,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 567.500161] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:02.170Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7e743e8bed3f676f10d26db2","created_at":"2026-11-07T11:07:59.859Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39da05def194a7bba2cfe2e9a822a41301d489712b85a02b788cc015e1a891ff","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:59.869Z","job_size_bytes":467,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7e743e8bed3f676f10d26db2: start","job_status":"start","queue_duration_s":2.300633,"scheduling_latency_s":2.300672} vm-test-run-nixbot-gitlab> gitlab # [ 567.517894] gitaly[928]: time="2026-11-07T11:08:02.218Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:32.117" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:02.217" grpc.time_ms=0.829 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.530863] gitaly[928]: time="2026-11-07T11:08:02.236Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-07T11:08:12.142" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:02.223" grpc.time_ms=12.933 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.692876] gitaly[928]: time="2026-11-07T11:08:02.416Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:32.315" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:02.415" grpc.time_ms=1.056 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.713305] gitaly[928]: time="2026-11-07T11:08:02.426Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=47 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-07T11:08:32.322" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:02.422" grpc.time_ms=3.426 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 567.846432] gitaly[928]: time="2026-11-07T11:08:02.570Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:12.148" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:02.558" grpc.time_ms=11.734 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 568.433446] nixbot[4361]: {"ts": "2026-11-07T11:08:03.157Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43232 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.04 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 568.766499] gitaly[928]: time="2026-11-07T11:08:03.489Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-07T11:08:33.368" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:03.468" grpc.time_ms=20.573 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 568.792593] gitaly[928]: time="2026-11-07T11:08:03.515Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6TTZTAKCDE7VHFDCCCEWX3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-07T11:08:33.394" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:03.494" grpc.time_ms=20.307 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 568.834664] sidekiq-cluster[4230]: {"severity":"WARN","time":"2026-11-07T11:08:03.557Z","class":"CreatePipelineWorker","correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"4e706c3ecda168d9d017ec091fafdbae76772037","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4e706c3ecda168d9d017ec091fafdbae76772037","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 568.846616] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7e743e8bed3f676f10d26db2","created_at":"2026-11-07T11:07:59.859Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39da05def194a7bba2cfe2e9a822a41301d489712b85a02b788cc015e1a891ff","size_limiter":"validated","enqueued_at":"2026-11-07T11:07:59.869Z","job_size_bytes":467,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7e743e8bed3f676f10d26db2: done: 1.396594 sec","job_status":"done","queue_duration_s":2.300633,"scheduling_latency_s":2.300672,"gitaly_calls":7,"gitaly_duration_s":0.09505,"redis_calls":51,"redis_duration_s":0.013939,"redis_read_bytes":169,"redis_write_bytes":4166,"redis_cache_calls":4,"redis_cache_duration_s":0.00118,"redis_cache_read_bytes":10,"redis_cache_write_bytes":341,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006429,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003591,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.233408,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035594,"duration_s":1.396594,"completed_at":"2026-11-07T11:08:03.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075578,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 568.925128] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.568Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"fc24eed6-d318-44e9-923f-d27ec3adc734\"}"],"class":"WebHookWorker","jid":"0c14f8ea7d7146cc6af8e9ba","created_at":"2026-11-07T11:08:00.039Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:db27d28c5a58c941610ce883f0edb97597b31aedbeba5dcd7154ef5dc0d5a4d1","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:00.049Z","job_size_bytes":1751,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0c14f8ea7d7146cc6af8e9ba: start","job_status":"start","queue_duration_s":3.519817,"scheduling_latency_s":3.519843} vm-test-run-nixbot-gitlab> gitlab # [ 569.051529] nixbot[4361]: {"ts": "2026-11-07T11:08:03.774Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "4d1fc646-ca8b-459d-b2e5-86f4d388f005", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "4e706c3ecda168d9d017ec091fafdbae76772037", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 569.058975] nixbot[4361]: {"ts": "2026-11-07T11:08:03.782Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43240 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} vm-test-run-nixbot-gitlab> gitlab # [ 569.093811] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.816Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"fc24eed6-d318-44e9-923f-d27ec3adc734\"}"],"class":"WebHookWorker","jid":"0c14f8ea7d7146cc6af8e9ba","created_at":"2026-11-07T11:08:00.039Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-c6f87fb1db3d4a1d","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:db27d28c5a58c941610ce883f0edb97597b31aedbeba5dcd7154ef5dc0d5a4d1","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:00.049Z","job_size_bytes":1751,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0c14f8ea7d7146cc6af8e9ba: done: 0.247951 sec","job_status":"done","queue_duration_s":3.519817,"scheduling_latency_s":3.519843,"redis_calls":19,"redis_duration_s":0.005822,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001453,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000742,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00231,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001317,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09161294000000453,"cpu_s":0.138098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048926,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.247951,"completed_at":"2026-11-07T11:08:03.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.168651] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.820Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9da0413cc0f553ff3cce69b2","created_at":"2026-11-07T11:08:00.894Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:00.902Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9da0413cc0f553ff3cce69b2: start","job_status":"start","queue_duration_s":2.918039,"scheduling_latency_s":2.918077} vm-test-run-nixbot-gitlab> gitlab # [ 569.208232] gitaly[928]: time="2026-11-07T11:08:03.932Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CBHVC419D3DV96D681N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:03.915" grpc.time_ms=16.502 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.229570] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 106 vm-test-run-nixbot-gitlab> gitlab # [ 569.268609] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.991Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9da0413cc0f553ff3cce69b2","created_at":"2026-11-07T11:08:00.894Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:00.902Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9da0413cc0f553ff3cce69b2: done: 0.171145 sec","job_status":"done","queue_duration_s":2.918039,"scheduling_latency_s":2.918077,"redis_calls":10,"redis_duration_s":0.0033480000000000003,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000722,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000747,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063252,"duration_s":0.171145,"completed_at":"2026-11-07T11:08:03.991Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042354,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 569.335770] gitaly[928]: time="2026-11-07T11:08:04.055Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CEY94NXQYPEVY2SGB74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.042" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.347320] gitlab-workhorse[830]: time="2026-11-07T11:08:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 569.349548] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 121 vm-test-run-nixbot-gitlab> gitlab # [ 569.351593] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:03.994Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d38956ff3ce2dfed620031d3","created_at":"2026-11-07T11:08:01.089Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.098Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d38956ff3ce2dfed620031d3: start","job_status":"start","queue_duration_s":2.895793,"scheduling_latency_s":2.895833} vm-test-run-nixbot-gitlab> gitlab # [ 569.368999] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.063Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d38956ff3ce2dfed620031d3","created_at":"2026-11-07T11:08:01.089Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.098Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d38956ff3ce2dfed620031d3: done: 0.068994 sec","job_status":"done","queue_duration_s":2.895793,"scheduling_latency_s":2.895833,"redis_calls":10,"redis_duration_s":0.003894,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054734,"duration_s":0.068994,"completed_at":"2026-11-07T11:08:04.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.417142] gitaly[928]: time="2026-11-07T11:08:04.055Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CEY94NXQYPEVY2SGB74 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.042" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 569.430978] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.066Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"04111cf4fcaef6f6583182e2","created_at":"2026-11-07T11:08:01.121Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.134Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-04111cf4fcaef6f6583182e2: start","job_status":"start","queue_duration_s":2.93142,"scheduling_latency_s":2.931457} vm-test-run-nixbot-gitlab> gitlab # [ 569.447000] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.141Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"04111cf4fcaef6f6583182e2","created_at":"2026-11-07T11:08:01.121Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.134Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-04111cf4fcaef6f6583182e2: done: 0.075316 sec","job_status":"done","queue_duration_s":2.93142,"scheduling_latency_s":2.931457,"redis_calls":10,"redis_duration_s":0.0059229999999999994,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000933,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046984,"duration_s":0.075316,"completed_at":"2026-11-07T11:08:04.141Z","load_balancing_strategy":"primary","db_duration_s":0.010885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.495257] gitaly[928]: time="2026-11-07T11:08:04.055Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CEY94NXQYPEVY2SGB74 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.042" grpc.time_ms=12.585 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.510789] gitlab-workhorse[830]: time="2026-11-07T11:08:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 vm-test-run-nixbot-gitlab> gitlab # [ 569.513125] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 124 vm-test-run-nixbot-gitlab> gitlab # [ 569.515198] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fb706b95559b2bcc26f51f71","created_at":"2026-11-07T11:08:01.159Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.169Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb706b95559b2bcc26f51f71: start","job_status":"start","queue_duration_s":2.978,"scheduling_latency_s":2.978042} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.10 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 569.532168] gitaly[928]: time="2026-11-07T11:08:04.181Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CJTV094F6AJW5SK5Z7D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.160" pid=928 remote_ip="2001:db8:1::1" request_sha=8e2fe50d9c2c379775992a4611ed1fb4cdcfab5c response_bytes=337 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.543459] nixbot[4361]: {"ts": "2026-11-07T11:08:04.222Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} vm-test-run-nixbot-gitlab> gitlab # [ 569.546028] nixbot[4361]: {"ts": "2026-11-07T11:08:04.225Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "4e706c3ecda168d9d017ec091fafdbae76772037"} vm-test-run-nixbot-gitlab> gitlab # [ 569.548487] nixbot[4361]: {"ts": "2026-11-07T11:08:04.250Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43254 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 569.551373] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fb706b95559b2bcc26f51f71","created_at":"2026-11-07T11:08:01.159Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.169Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb706b95559b2bcc26f51f71: done: 0.097421 sec","job_status":"done","queue_duration_s":2.978,"scheduling_latency_s":2.978042,"redis_calls":13,"redis_duration_s":0.008011,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000885,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004551,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0012,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00109506,"duration_s":0.097421,"completed_at":"2026-11-07T11:08:04.244Z","load_balancing_strategy":"primary","db_duration_s":0.012404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.598468] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.250Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8fb32cdd3ab58be742e9a7d9","created_at":"2026-11-07T11:08:01.170Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.179Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8fb32cdd3ab58be742e9a7d9: start","job_status":"start","queue_duration_s":3.070932,"scheduling_latency_s":3.070975} vm-test-run-nixbot-gitlab> gitlab # [ 569.622833] gitaly[928]: time="2026-11-07T11:08:04.181Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CJTV094F6AJW5SK5Z7D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.160" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.637277] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8fb32cdd3ab58be742e9a7d9","created_at":"2026-11-07T11:08:01.170Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.179Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8fb32cdd3ab58be742e9a7d9: done: 0.084282 sec","job_status":"done","queue_duration_s":3.070932,"scheduling_latency_s":3.070975,"redis_calls":10,"redis_duration_s":0.003813,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050896,"duration_s":0.084282,"completed_at":"2026-11-07T11:08:04.334Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.683984] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9cbdbccfdb33350044d512ce","created_at":"2026-11-07T11:08:01.180Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.187Z","job_size_bytes":19,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9cbdbccfdb33350044d512ce: start","job_status":"start","queue_duration_s":3.150453,"scheduling_latency_s":3.150502} vm-test-run-nixbot-gitlab> gitlab # [ 569.700795] gitaly[928]: time="2026-11-07T11:08:04.182Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CJTV094F6AJW5SK5Z7D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:04.160" grpc.time_ms=21.457 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 569.716197] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9cbdbccfdb33350044d512ce","created_at":"2026-11-07T11:08:01.180Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.187Z","job_size_bytes":19,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9cbdbccfdb33350044d512ce: done: 0.073442 sec","job_status":"done","queue_duration_s":3.150453,"scheduling_latency_s":3.150502,"redis_calls":11,"redis_duration_s":0.004055,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00072,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002627,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042592,"duration_s":0.073442,"completed_at":"2026-11-07T11:08:04.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.761811] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.414Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c06d36ebf7516840963a4f28","created_at":"2026-11-07T11:08:01.191Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.198Z","job_size_bytes":66,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c06d36ebf7516840963a4f28: start","job_status":"start","queue_duration_s":3.2163,"scheduling_latency_s":3.216341} vm-test-run-nixbot-gitlab> gitlab # [ 569.780068] gitaly[928]: time="2026-11-07T11:08:04.344Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:34.342" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:04.343" grpc.time_ms=0.86 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 569.793341] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.486Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c06d36ebf7516840963a4f28","created_at":"2026-11-07T11:08:01.191Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.198Z","job_size_bytes":66,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c06d36ebf7516840963a4f28: done: 0.071583 sec","job_status":"done","queue_duration_s":3.2163,"scheduling_latency_s":3.216341,"redis_calls":10,"redis_duration_s":0.004189,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001617,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006609,"duration_s":0.071583,"completed_at":"2026-11-07T11:08:04.486Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.858408] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2ff6a60eee78b4d87080fb73","created_at":"2026-11-07T11:08:01.200Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.206Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2ff6a60eee78b4d87080fb73: start","job_status":"start","queue_duration_s":3.282548,"scheduling_latency_s":3.282588} vm-test-run-nixbot-gitlab> gitlab # [ 569.876380] gitaly[928]: time="2026-11-07T11:08:04.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:14.064" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:04.455" grpc.time_ms=16.499 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 569.890577] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 339 vm-test-run-nixbot-gitlab> gitlab # [ 569.893363] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2ff6a60eee78b4d87080fb73","created_at":"2026-11-07T11:08:01.200Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.206Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2ff6a60eee78b4d87080fb73: done: 0.091843 sec","job_status":"done","queue_duration_s":3.282548,"scheduling_latency_s":3.282588,"redis_calls":10,"redis_duration_s":0.0038529999999999997,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00138,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001756,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046514,"duration_s":0.091843,"completed_at":"2026-11-07T11:08:04.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 569.939424] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.583Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"08e69866a67292d3b2b0ba70","created_at":"2026-11-07T11:08:01.215Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.221Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-08e69866a67292d3b2b0ba70: start","job_status":"start","queue_duration_s":3.361724,"scheduling_latency_s":3.36176} vm-test-run-nixbot-gitlab> gitlab # [ 569.956585] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.672Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"08e69866a67292d3b2b0ba70","created_at":"2026-11-07T11:08:01.215Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.221Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-08e69866a67292d3b2b0ba70: done: 0.089178 sec","job_status":"done","queue_duration_s":3.361724,"scheduling_latency_s":3.36176,"redis_calls":10,"redis_duration_s":0.007062000000000001,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002667,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086804,"duration_s":0.089178,"completed_at":"2026-11-07T11:08:04.672Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 570.002483] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"776639afb5b55de03576afd5","created_at":"2026-11-07T11:08:01.222Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.227Z","job_size_bytes":6,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-776639afb5b55de03576afd5: start","job_status":"start","queue_duration_s":3.448116,"scheduling_latency_s":3.448162} vm-test-run-nixbot-gitlab> gitlab # [ 570.019017] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.724Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"776639afb5b55de03576afd5","created_at":"2026-11-07T11:08:01.222Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.227Z","job_size_bytes":6,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-776639afb5b55de03576afd5: done: 0.048694 sec","job_status":"done","queue_duration_s":3.448116,"scheduling_latency_s":3.448162,"redis_calls":10,"redis_duration_s":0.003322,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001857,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049804,"duration_s":0.048694,"completed_at":"2026-11-07T11:08:04.724Z","load_balancing_strategy":"primary","db_duration_s":0.003923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 570.065386] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.729Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"be12c478947d3f0814ab50ea","created_at":"2026-11-07T11:08:01.228Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.233Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-be12c478947d3f0814ab50ea: start","job_status":"start","queue_duration_s":3.496057,"scheduling_latency_s":3.496095} vm-test-run-nixbot-gitlab> gitlab # [ 570.081461] gitaly[928]: time="2026-11-07T11:08:04.719Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3D3Q7B813D566Z5CH9TD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:34.318" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:04.718" grpc.time_ms=0.708 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.096960] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.804Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"be12c478947d3f0814ab50ea","created_at":"2026-11-07T11:08:01.228Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.233Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-be12c478947d3f0814ab50ea: done: 0.074959 sec","job_status":"done","queue_duration_s":3.496057,"scheduling_latency_s":3.496095,"redis_calls":10,"redis_duration_s":0.004403,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000643,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001636,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007211,"duration_s":0.074959,"completed_at":"2026-11-07T11:08:04.804Z","load_balancing_strategy":"primary","db_duration_s":0.012371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 570.141938] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.807Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9072b362c15c8938f82cda89","created_at":"2026-11-07T11:08:01.241Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.248Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9072b362c15c8938f82cda89: start","job_status":"start","queue_duration_s":3.558345,"scheduling_latency_s":3.558382} vm-test-run-nixbot-gitlab> gitlab # [ 570.159103] gitaly[928]: time="2026-11-07T11:08:04.824Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3D3Q7B813D566Z5CH9TD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:14.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:04.812" grpc.time_ms=11.706 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.185589] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 309 vm-test-run-nixbot-gitlab> gitlab # [ 570.198227] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.921Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9072b362c15c8938f82cda89","created_at":"2026-11-07T11:08:01.241Z","trace_propagation_headers":{"sentry-trace":"be006074cdec473f93ba28198d10b6a1-bcbb4053b8d1473c","baggage":"sentry-trace_id=be006074cdec473f93ba28198d10b6a1,sentry-release=Unknown"},"correlation_id":"01M6TV011KN200XPJ4BA8PBQPS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.248Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9072b362c15c8938f82cda89: done: 0.114547 sec","job_status":"done","queue_duration_s":3.558345,"scheduling_latency_s":3.558382,"redis_calls":14,"redis_duration_s":0.0056159999999999995,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000824,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00263,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00072698,"duration_s":0.114547,"completed_at":"2026-11-07T11:08:04.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 570.246080] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:04.923Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9d762f2cb29bd8a000cacfe6","created_at":"2026-11-07T11:08:01.472Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"60f9b823a38c5b8b02ffb481","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.477Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9d762f2cb29bd8a000cacfe6: start","job_status":"start","queue_duration_s":3.446585,"scheduling_latency_s":3.446618} vm-test-run-nixbot-gitlab> gitlab # [ 570.263933] gitaly[928]: time="2026-11-07T11:08:04.970Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF diskcache=8b0c3c8c-bb3a-452d-b4fd-44ba258a371a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-07T17:08:04.953" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:04.954" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.274409] gitaly[928]: time="2026-11-07T11:08:04.971Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-07T17:08:04.953" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:04.954" grpc.time_ms=16.664 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.288219] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:05.001Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9d762f2cb29bd8a000cacfe6","created_at":"2026-11-07T11:08:01.472Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"60f9b823a38c5b8b02ffb481","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.477Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9d762f2cb29bd8a000cacfe6: done: 0.077832 sec","job_status":"done","queue_duration_s":3.446585,"scheduling_latency_s":3.446618,"gitaly_calls":1,"gitaly_duration_s":0.020321,"redis_calls":12,"redis_duration_s":0.0038079999999999998,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001401,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000835,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04806694,"duration_s":0.077832,"completed_at":"2026-11-07T11:08:05.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00975,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 570.335512] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:05.003Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0bd11f67b5769061d930a0a6","created_at":"2026-11-07T11:08:01.659Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:622439c7a667fdeb888b4c0f9dd4634433fbad8dd9855b4e0914188b8fa23446","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.665Z","job_size_bytes":467,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0bd11f67b5769061d930a0a6: start","job_status":"start","queue_duration_s":3.338606,"scheduling_latency_s":3.338637} vm-test-run-nixbot-gitlab> gitlab # [ 570.353549] gitaly[928]: time="2026-11-07T11:08:05.016Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3DDPNQ62AAJ8PBGXTREG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:35.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:05.015" grpc.time_ms=1.004 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.366744] gitaly[928]: time="2026-11-07T11:08:05.044Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:35.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:05.043" grpc.time_ms=0.819 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.379596] gitaly[928]: time="2026-11-07T11:08:05.059Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-07T11:08:15.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:05.049" grpc.time_ms=9.778 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.393238] gitaly[928]: time="2026-11-07T11:08:05.103Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3DDPNQ62AAJ8PBGXTREG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:15.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:05.087" grpc.time_ms=15.849 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.407207] gitaly[928]: time="2026-11-07T11:08:05.120Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:35.018" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:05.118" grpc.time_ms=1.192 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.420576] gitaly[928]: time="2026-11-07T11:08:05.132Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=51 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-07T11:08:35.025" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:05.125" grpc.time_ms=6.544 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.470862] gitaly[928]: time="2026-11-07T11:08:05.194Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:15.139" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:05.180" grpc.time_ms=14.248 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 570.485903] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4e706c3ecda168d9d017ec091fafdbae76772037 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 276 vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.07 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 570.619236] nixbot[4361]: {"ts": "2026-11-07T11:08:05.341Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43260 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 570.834616] gitaly[928]: time="2026-11-07T11:08:05.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3DQT4MTTJS6Q2ZXB0F0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.541" grpc.time_ms=15.873 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 570.857814] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:05 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 316 vm-test-run-nixbot-gitlab> gitlab # [ 570.959606] gitaly[928]: time="2026-11-07T11:08:05.683Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3E1RDXX01FXS7BCMN1PT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.669" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 570.976962] gitaly[928]: time="2026-11-07T11:08:05.683Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3E1RDXX01FXS7BCMN1PT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.669" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 570.999050] gitlab-workhorse[830]: time="2026-11-07T11:08:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 571.001945] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 124 vm-test-run-nixbot-gitlab> gitlab # [ 571.006495] gitaly[928]: time="2026-11-07T11:08:05.684Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3E1RDXX01FXS7BCMN1PT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.669" grpc.time_ms=14.438 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.126499] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:05 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 118 vm-test-run-nixbot-gitlab> gitlab # [ 571.132257] gitaly[928]: time="2026-11-07T11:08:05.849Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3E73ERS8QPTNSKTB0H7X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.829" grpc.time_ms=20.13 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.261886] gitaly[928]: time="2026-11-07T11:08:05.985Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EAWVZMGS0CQ44BVZ7MX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.970" pid=928 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.275779] gitaly[928]: time="2026-11-07T11:08:05.986Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EAWVZMGS0CQ44BVZ7MX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.970" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.291259] gitlab-workhorse[830]: time="2026-11-07T11:08:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 571.293445] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 134 vm-test-run-nixbot-gitlab> gitlab # [ 571.295444] gitaly[928]: time="2026-11-07T11:08:05.986Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EAWVZMGS0CQ44BVZ7MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:05.970" grpc.time_ms=15.289 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.310780] gitaly[928]: time="2026-11-07T11:08:06.026Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-07T11:08:36.006" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.006" grpc.time_ms=19.305 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.326695] gitaly[928]: time="2026-11-07T11:08:06.047Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV33P8EXV4540VK1ZFAPDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-07T11:08:36.031" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.031" grpc.time_ms=16.334 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.347440] sidekiq-cluster[4230]: {"severity":"WARN","time":"2026-11-07T11:08:06.070Z","class":"CreatePipelineWorker","correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"4e706c3ecda168d9d017ec091fafdbae76772037","after":"df7e7ae3022955d8a907d68546822b663a3d4ff9","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"df7e7ae3022955d8a907d68546822b663a3d4ff9","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 571.357276] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0bd11f67b5769061d930a0a6","created_at":"2026-11-07T11:08:01.659Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:622439c7a667fdeb888b4c0f9dd4634433fbad8dd9855b4e0914188b8fa23446","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.665Z","job_size_bytes":467,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0bd11f67b5769061d930a0a6: done: 1.076658 sec","job_status":"done","queue_duration_s":3.338606,"scheduling_latency_s":3.338637,"gitaly_calls":7,"gitaly_duration_s":0.095365,"redis_calls":33,"redis_duration_s":0.008882,"redis_read_bytes":184,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001034,"redis_cache_read_bytes":43,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004887,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.950046,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.076658,"completed_at":"2026-11-07T11:08:06.080Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03541,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 571.406646] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.083Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8af5fcfa-6557-4e0c-9315-6edec6de5fe2\"}"],"class":"WebHookWorker","jid":"fcaab63d3cd7903ae3883f35","created_at":"2026-11-07T11:08:01.695Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a0ed26d95654de31eec029261a2b8fb878cfa7869ef763163a34df2a9e4574cf","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.701Z","job_size_bytes":1749,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fcaab63d3cd7903ae3883f35: start","job_status":"start","queue_duration_s":4.381654,"scheduling_latency_s":4.381687} vm-test-run-nixbot-gitlab> gitlab # [ 571.424868] nixbot[4361]: {"ts": "2026-11-07T11:08:06.127Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "5a237960-ec70-4b86-a941-41b5fc75f00a", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "df7e7ae3022955d8a907d68546822b663a3d4ff9", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 571.428693] nixbot[4361]: {"ts": "2026-11-07T11:08:06.133Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:44948 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} vm-test-run-nixbot-gitlab> gitlab # [ 571.430815] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.150Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8af5fcfa-6557-4e0c-9315-6edec6de5fe2\"}"],"class":"WebHookWorker","jid":"fcaab63d3cd7903ae3883f35","created_at":"2026-11-07T11:08:01.695Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-eb5a9850f09544be","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a0ed26d95654de31eec029261a2b8fb878cfa7869ef763163a34df2a9e4574cf","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.701Z","job_size_bytes":1749,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fcaab63d3cd7903ae3883f35: done: 0.067082 sec","job_status":"done","queue_duration_s":4.381654,"scheduling_latency_s":4.381687,"redis_calls":14,"redis_duration_s":0.003933,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000676,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00198,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001277,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.013878620000014052,"cpu_s":0.040772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.067082,"completed_at":"2026-11-07T11:08:06.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 571.480935] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"dc43396d397d95694dfa59cc","created_at":"2026-11-07T11:08:01.895Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-6a4ee43dcb3b4189","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.905Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-dc43396d397d95694dfa59cc: start","job_status":"start","queue_duration_s":4.248455,"scheduling_latency_s":4.248491} vm-test-run-nixbot-gitlab> gitlab # [ 571.497560] gitaly[928]: time="2026-11-07T11:08:06.168Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EH1KG5FHQ6JC8DEXNJP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.154" grpc.time_ms=14.41 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.511734] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 120 vm-test-run-nixbot-gitlab> gitlab # [ 571.564589] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.287Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"dc43396d397d95694dfa59cc","created_at":"2026-11-07T11:08:01.895Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-6a4ee43dcb3b4189","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:01.905Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-dc43396d397d95694dfa59cc: done: 0.13367 sec","job_status":"done","queue_duration_s":4.248455,"scheduling_latency_s":4.248491,"redis_calls":19,"redis_duration_s":0.009831,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004566,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000923,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003067,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.017160340000000003,"duration_s":0.13367,"completed_at":"2026-11-07T11:08:06.287Z","load_balancing_strategy":"primary","db_duration_s":0.018173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 571.622138] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.291Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0c14f8ea7d7146cc6af8e9ba"],"class":"WebHooks::LogExecutionWorker","jid":"5d9d603f373cbd6f39fca0d5","created_at":"2026-11-07T11:08:03.788Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-50ae839f4cd64110","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:972a5bb4e6eb0ce1898cd11930d769d08a0e3e57960b386097b5ccc82594e368","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:03.801Z","job_size_bytes":2593,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5d9d603f373cbd6f39fca0d5: start","job_status":"start","queue_duration_s":2.489668,"scheduling_latency_s":2.489711} vm-test-run-nixbot-gitlab> gitlab # [ 571.645876] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 140 vm-test-run-nixbot-gitlab> gitlab # [ 571.648145] gitlab-workhorse[830]: time="2026-11-07T11:08:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 vm-test-run-nixbot-gitlab> gitlab # [ 571.650057] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 151 vm-test-run-nixbot-gitlab> gitlab # [ 571.652065] gitaly[928]: time="2026-11-07T11:08:06.310Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EMYDZ3BH87H8XE4H750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.291" grpc.time_ms=18.543 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.665701] gitaly[928]: time="2026-11-07T11:08:06.324Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EMX3PBHKHV2H1CQ7V0Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.292" pid=928 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.676443] gitaly[928]: time="2026-11-07T11:08:06.324Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EMX3PBHKHV2H1CQ7V0Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.292" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.09 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 571.691981] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.382Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0c14f8ea7d7146cc6af8e9ba"],"class":"WebHooks::LogExecutionWorker","jid":"5d9d603f373cbd6f39fca0d5","created_at":"2026-11-07T11:08:03.788Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-50ae839f4cd64110","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:972a5bb4e6eb0ce1898cd11930d769d08a0e3e57960b386097b5ccc82594e368","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:03.801Z","job_size_bytes":2593,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5d9d603f373cbd6f39fca0d5: done: 0.090914 sec","job_status":"done","queue_duration_s":2.489668,"scheduling_latency_s":2.489711,"redis_calls":10,"redis_duration_s":0.0044930000000000005,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000819,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000811,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038756,"duration_s":0.090914,"completed_at":"2026-11-07T11:08:06.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 571.740079] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.384Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3e6baa5d9b5ec4abfc1cd32d","created_at":"2026-11-07T11:08:04.547Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-3feac5571fee40a4","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:04.555Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3e6baa5d9b5ec4abfc1cd32d: start","job_status":"start","queue_duration_s":1.828998,"scheduling_latency_s":1.829028} vm-test-run-nixbot-gitlab> gitlab # [ 571.757093] nixbot[4361]: {"ts": "2026-11-07T11:08:06.414Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:44962 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 571.760172] gitaly[928]: time="2026-11-07T11:08:06.324Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EMX3PBHKHV2H1CQ7V0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.292" grpc.time_ms=31.908 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.776165] gitlab-workhorse[830]: time="2026-11-07T11:08:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 571.778143] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 140 vm-test-run-nixbot-gitlab> gitlab # [ 571.780113] gitaly[928]: time="2026-11-07T11:08:06.457Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3ESDWSTCFWV7MW0N9NAV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.442" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.790650] gitaly[928]: time="2026-11-07T11:08:06.457Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3ESDWSTCFWV7MW0N9NAV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.442" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.804592] gitaly[928]: time="2026-11-07T11:08:06.458Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3ESDWSTCFWV7MW0N9NAV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:06.442" grpc.time_ms=15.752 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.854856] gitaly[928]: time="2026-11-07T11:08:06.578Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:06.577" grpc.time_ms=0.519 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.877619] gitaly[928]: time="2026-11-07T11:08:06.601Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:36.199" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.600" grpc.time_ms=0.897 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.895331] gitaly[928]: time="2026-11-07T11:08:06.608Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:06.607" grpc.time_ms=0.744 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.909659] gitaly[928]: time="2026-11-07T11:08:06.633Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-07T11:08:36.219" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:06.619" grpc.time_ms=13.616 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.926003] gitaly[928]: time="2026-11-07T11:08:06.646Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:06.645" grpc.time_ms=0.609 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 571.938119] gitaly[928]: time="2026-11-07T11:08:06.660Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:16.254" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:06.646" grpc.time_ms=13.63 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.953042] gitaly[928]: time="2026-11-07T11:08:06.660Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V diskcache=645e8fbb-afaa-40a1-8778-2bbd7b293460 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:36.440" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:06.541" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.963148] gitaly[928]: time="2026-11-07T11:08:06.661Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=108 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7259 command.oublock=0 command.real_time_ms=111 command.spawn_token_fork_ms=1 command.system_time_ms=63 command.user_time_ms=45 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:36.440" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:06.541" grpc.time_ms=119.365 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.977873] gitaly[928]: time="2026-11-07T11:08:06.692Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-07T11:08:16.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:08:06.692" grpc.time_ms=0.384 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 571.989076] gitaly[928]: time="2026-11-07T11:08:06.710Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-07T11:08:16.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:06.701" grpc.time_ms=9.701 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.043229] gitaly[928]: time="2026-11-07T11:08:06.767Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:36.665" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.765" grpc.time_ms=1.142 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.061231] gitaly[928]: time="2026-11-07T11:08:06.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:06.784" grpc.time_ms=1.028 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 572.093296] gitaly[928]: time="2026-11-07T11:08:06.817Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:06.816" grpc.time_ms=0.707 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 572.107398] gitaly[928]: time="2026-11-07T11:08:06.828Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN diskcache=46177757-79d6-477c-9a6e-78893499bca3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:08:16.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:08:06.715" pid=928 ref=refs/keep-around/df7e7ae3022955d8a907d68546822b663a3d4ff9 remote_ip="2001:db8:1::1" revision=df7e7ae3022955d8a907d68546822b663a3d4ff9 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.120710] gitaly[928]: time="2026-11-07T11:08:06.829Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5224 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=1 command.system_time_ms=74 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-07T11:08:16.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-07T11:08:06.715" grpc.time_ms=113.817 pid=928 ref=refs/keep-around/df7e7ae3022955d8a907d68546822b663a3d4ff9 remote_ip="2001:db8:1::1" revision=df7e7ae3022955d8a907d68546822b663a3d4ff9 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.136691] gitaly[928]: time="2026-11-07T11:08:06.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:36.435" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.836" grpc.time_ms=0.727 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.149972] gitaly[928]: time="2026-11-07T11:08:06.852Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3CRMXAV887QBT536HZ1V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:36.749" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:06.850" grpc.time_ms=1.34 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 572.163052] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.874Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3e6baa5d9b5ec4abfc1cd32d","created_at":"2026-11-07T11:08:04.547Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-3feac5571fee40a4","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:04.555Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":4230,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3e6baa5d9b5ec4abfc1cd32d: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 572.179011] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.888Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3e6baa5d9b5ec4abfc1cd32d","created_at":"2026-11-07T11:08:04.547Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-3feac5571fee40a4","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:04.555Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3e6baa5d9b5ec4abfc1cd32d: done: 0.503981 sec","job_status":"done","queue_duration_s":1.828998,"scheduling_latency_s":1.829028,"gitaly_calls":3,"gitaly_duration_s":0.134441,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.023854,"redis_read_bytes":148,"redis_write_bytes":18012,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001318,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001327,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004435,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12311,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.007352,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000484,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002474,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.005146,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.085,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282573,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.409697,"duration_s":0.503981,"completed_at":"2026-11-07T11:08:06.888Z","load_balancing_strategy":"primary","db_duration_s":0.087152,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 572.232996] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.890Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fcaab63d3cd7903ae3883f35"],"class":"WebHooks::LogExecutionWorker","jid":"9632dfb3818c6beba86b6217","created_at":"2026-11-07T11:08:06.135Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-116935c97f2a4b90","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2601731d19c7de6785af885325b0e1f83f084dab27b4ef5563f0cb5fb7713add","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.139Z","job_size_bytes":2592,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9632dfb3818c6beba86b6217: start","job_status":"start","queue_duration_s":0.750656,"scheduling_latency_s":0.750707} vm-test-run-nixbot-gitlab> gitlab # [ 572.253115] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.937Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fcaab63d3cd7903ae3883f35"],"class":"WebHooks::LogExecutionWorker","jid":"9632dfb3818c6beba86b6217","created_at":"2026-11-07T11:08:06.135Z","trace_propagation_headers":{"sentry-trace":"d29cc00fb1754195b33602f60a688b6e-116935c97f2a4b90","baggage":"sentry-trace_id=d29cc00fb1754195b33602f60a688b6e,sentry-release=Unknown"},"correlation_id":"01M6TV33P8EXV4540VK1ZFAPDF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2601731d19c7de6785af885325b0e1f83f084dab27b4ef5563f0cb5fb7713add","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.139Z","job_size_bytes":2592,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9632dfb3818c6beba86b6217: done: 0.047303 sec","job_status":"done","queue_duration_s":0.750656,"scheduling_latency_s":0.750707,"redis_calls":7,"redis_duration_s":0.002626,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047303,"completed_at":"2026-11-07T11:08:06.937Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.298292] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:06.940Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f17d7d09c7d722be3c33502e","created_at":"2026-11-07T11:08:06.257Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-2a0301bdbdac4edf","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.268Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f17d7d09c7d722be3c33502e: start","job_status":"start","queue_duration_s":0.671633,"scheduling_latency_s":0.671673} vm-test-run-nixbot-gitlab> gitlab # [ 572.315972] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 520 vm-test-run-nixbot-gitlab> gitlab # [ 572.318805] nixbot[4361]: {"ts": "2026-11-07T11:08:07.037Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-df7e7ae30229-6"} vm-test-run-nixbot-gitlab> gitlab # [ 572.397731] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.120Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f17d7d09c7d722be3c33502e","created_at":"2026-11-07T11:08:06.257Z","trace_propagation_headers":{"sentry-trace":"ff05c168e68349a38f7ca83dc5ba12bf-2a0301bdbdac4edf","baggage":"sentry-trace_id=ff05c168e68349a38f7ca83dc5ba12bf,sentry-release=Unknown"},"correlation_id":"01M6TTZTAKCDE7VHFDCCCEWX3T","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.268Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f17d7d09c7d722be3c33502e: done: 0.180106 sec","job_status":"done","queue_duration_s":0.671633,"scheduling_latency_s":0.671673,"redis_calls":11,"redis_duration_s":0.0038789999999999996,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000658,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002469,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.136141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005969,"duration_s":0.180106,"completed_at":"2026-11-07T11:08:07.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.465136] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a8e9214fd69e669d4fbedde7","created_at":"2026-11-07T11:08:06.525Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.530Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a8e9214fd69e669d4fbedde7: start","job_status":"start","queue_duration_s":0.593957,"scheduling_latency_s":0.594008} vm-test-run-nixbot-gitlab> gitlab # [ 572.498009] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a8e9214fd69e669d4fbedde7","created_at":"2026-11-07T11:08:06.525Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.530Z","job_size_bytes":23,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a8e9214fd69e669d4fbedde7: done: 0.09412 sec","job_status":"done","queue_duration_s":0.593957,"scheduling_latency_s":0.594008,"redis_calls":7,"redis_duration_s":0.003113,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000702,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.09412,"completed_at":"2026-11-07T11:08:07.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01655,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 572.553248] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.221Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"600db24693cbba7b24dbe5da","created_at":"2026-11-07T11:08:06.672Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.677Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-600db24693cbba7b24dbe5da: start","job_status":"start","queue_duration_s":0.544035,"scheduling_latency_s":0.544089} vm-test-run-nixbot-gitlab> gitlab # [ 572.571886] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.283Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"600db24693cbba7b24dbe5da","created_at":"2026-11-07T11:08:06.672Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.677Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-600db24693cbba7b24dbe5da: done: 0.062546 sec","job_status":"done","queue_duration_s":0.544035,"scheduling_latency_s":0.544089,"redis_calls":7,"redis_duration_s":0.006259000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003615,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062546,"completed_at":"2026-11-07T11:08:07.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.616837] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.286Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b4b005afaea1477a624f5956","created_at":"2026-11-07T11:08:06.696Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.702Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b4b005afaea1477a624f5956: start","job_status":"start","queue_duration_s":0.583263,"scheduling_latency_s":0.583301} vm-test-run-nixbot-gitlab> gitlab # [ 572.633000] nixbot[4361]: {"ts": "2026-11-07T11:08:07.240Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-df7e7ae30229-6", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 572.635503] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b4b005afaea1477a624f5956","created_at":"2026-11-07T11:08:06.696Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.702Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b4b005afaea1477a624f5956: done: 0.052087 sec","job_status":"done","queue_duration_s":0.583263,"scheduling_latency_s":0.583301,"redis_calls":7,"redis_duration_s":0.0028759999999999997,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000659,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052087,"completed_at":"2026-11-07T11:08:07.338Z","load_balancing_strategy":"primary","db_duration_s":0.008167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.680116] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e37820ed3ca5394f21d3bcfa","created_at":"2026-11-07T11:08:06.722Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.725Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e37820ed3ca5394f21d3bcfa: start","job_status":"start","queue_duration_s":0.615471,"scheduling_latency_s":0.615512} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9' vm-test-run-nixbot-gitlab> gitlab # [ 572.695740] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.391Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e37820ed3ca5394f21d3bcfa","created_at":"2026-11-07T11:08:06.722Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.725Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e37820ed3ca5394f21d3bcfa: done: 0.050763 sec","job_status":"done","queue_duration_s":0.615471,"scheduling_latency_s":0.615512,"redis_calls":10,"redis_duration_s":0.003268,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001054,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050763,"completed_at":"2026-11-07T11:08:07.391Z","load_balancing_strategy":"primary","db_duration_s":0.007839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.740915] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"400b0209c238c2588eb1d6e4","created_at":"2026-11-07T11:08:06.726Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.729Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-400b0209c238c2588eb1d6e4: start","job_status":"start","queue_duration_s":0.669382,"scheduling_latency_s":0.66942} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9', in 0.07 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 114.84 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 572.758266] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"400b0209c238c2588eb1d6e4","created_at":"2026-11-07T11:08:06.726Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.729Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-400b0209c238c2588eb1d6e4: done: 0.042546 sec","job_status":"done","queue_duration_s":0.669382,"scheduling_latency_s":0.66942,"redis_calls":7,"redis_duration_s":0.002352,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042546,"completed_at":"2026-11-07T11:08:07.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.802858] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5674096275f918a4a4e9eac5","created_at":"2026-11-07T11:08:06.730Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.734Z","job_size_bytes":19,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5674096275f918a4a4e9eac5: start","job_status":"start","queue_duration_s":0.709106,"scheduling_latency_s":0.709139} vm-test-run-nixbot-gitlab> gitlab # [ 572.819867] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.487Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5674096275f918a4a4e9eac5","created_at":"2026-11-07T11:08:06.730Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.734Z","job_size_bytes":19,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5674096275f918a4a4e9eac5: done: 0.044615 sec","job_status":"done","queue_duration_s":0.709106,"scheduling_latency_s":0.709139,"redis_calls":8,"redis_duration_s":0.005065999999999999,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004298,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044615,"completed_at":"2026-11-07T11:08:07.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.865100] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.491Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"37f36e54bb6c32323dee637f","created_at":"2026-11-07T11:08:06.735Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.738Z","job_size_bytes":66,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-37f36e54bb6c32323dee637f: start","job_status":"start","queue_duration_s":0.753459,"scheduling_latency_s":0.753496} vm-test-run-nixbot-gitlab> gitlab # [ 572.883732] nixbot[4361]: {"ts": "2026-11-07T11:08:07.503Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:44968 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1\" 200"} vm-test-run-nixbot-gitlab> gitlab # [ 572.887133] nix-daemon[4531]: accepted connection from pid 5761, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 572.888185] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.534Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"37f36e54bb6c32323dee637f","created_at":"2026-11-07T11:08:06.735Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.738Z","job_size_bytes":66,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-37f36e54bb6c32323dee637f: done: 0.042991 sec","job_status":"done","queue_duration_s":0.753459,"scheduling_latency_s":0.753496,"redis_calls":7,"redis_duration_s":0.003216,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042991,"completed_at":"2026-11-07T11:08:07.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 572.934033] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.539Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"de84e1b925fdc13c3d4733a7","created_at":"2026-11-07T11:08:06.738Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.741Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-de84e1b925fdc13c3d4733a7: start","job_status":"start","queue_duration_s":0.797573,"scheduling_latency_s":0.797613} vm-test-run-nixbot-gitlab> gitlab # [ 572.950925] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.579Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"de84e1b925fdc13c3d4733a7","created_at":"2026-11-07T11:08:06.738Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.741Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-de84e1b925fdc13c3d4733a7: done: 0.0402 sec","job_status":"done","queue_duration_s":0.797573,"scheduling_latency_s":0.797613,"redis_calls":7,"redis_duration_s":0.0030540000000000003,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0402,"completed_at":"2026-11-07T11:08:07.579Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.22 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 572.995323] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.583Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d06e79d20fb3aff006144864","created_at":"2026-11-07T11:08:06.742Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.744Z","job_size_bytes":6,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d06e79d20fb3aff006144864: start","job_status":"start","queue_duration_s":0.839065,"scheduling_latency_s":0.839094} vm-test-run-nixbot-gitlab> gitlab # [ 573.012230] gitaly[928]: time="2026-11-07T11:08:07.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3G0F0HS2KF26HTDFCT7C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:37.265" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:07.665" grpc.time_ms=2.929 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 573.025641] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 157 vm-test-run-nixbot-gitlab> gitlab # [ 573.028374] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.610Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d06e79d20fb3aff006144864","created_at":"2026-11-07T11:08:06.742Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.744Z","job_size_bytes":6,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d06e79d20fb3aff006144864: done: 0.027162 sec","job_status":"done","queue_duration_s":0.839065,"scheduling_latency_s":0.839094,"redis_calls":7,"redis_duration_s":0.009225,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007524,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001701,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027162,"completed_at":"2026-11-07T11:08:07.610Z","load_balancing_strategy":"primary","db_duration_s":0.000761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 573.071815] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"34498b4873a225dc7e8d4db5","created_at":"2026-11-07T11:08:06.745Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.747Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-34498b4873a225dc7e8d4db5: start","job_status":"start","queue_duration_s":0.867093,"scheduling_latency_s":0.867127} vm-test-run-nixbot-gitlab> gitlab # [ 573.087946] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.658Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"34498b4873a225dc7e8d4db5","created_at":"2026-11-07T11:08:06.745Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.747Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-34498b4873a225dc7e8d4db5: done: 0.043719 sec","job_status":"done","queue_duration_s":0.867093,"scheduling_latency_s":0.867127,"redis_calls":7,"redis_duration_s":0.010093,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043719,"completed_at":"2026-11-07T11:08:07.658Z","load_balancing_strategy":"primary","db_duration_s":0.005752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 573.132559] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.661Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2ac1e94f078ddfc7d6936739","created_at":"2026-11-07T11:08:06.840Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.849Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2ac1e94f078ddfc7d6936739: start","job_status":"start","queue_duration_s":0.812587,"scheduling_latency_s":0.812625} vm-test-run-nixbot-gitlab> gitlab # [ 573.149705] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.731Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2ac1e94f078ddfc7d6936739","created_at":"2026-11-07T11:08:06.840Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.849Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2ac1e94f078ddfc7d6936739: done: 0.070083 sec","job_status":"done","queue_duration_s":0.812587,"scheduling_latency_s":0.812625,"redis_calls":7,"redis_duration_s":0.003457,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070083,"completed_at":"2026-11-07T11:08:07.731Z","load_balancing_strategy":"primary","db_duration_s":0.020869,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 573.193858] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.734Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8e4cab90dea75a43363867a3","created_at":"2026-11-07T11:08:06.850Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.857Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8e4cab90dea75a43363867a3: start","job_status":"start","queue_duration_s":0.877736,"scheduling_latency_s":0.877771} vm-test-run-nixbot-gitlab> gitlab # [ 573.212208] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.825Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8e4cab90dea75a43363867a3","created_at":"2026-11-07T11:08:06.850Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.857Z","job_size_bytes":65,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8e4cab90dea75a43363867a3: done: 0.090255 sec","job_status":"done","queue_duration_s":0.877736,"scheduling_latency_s":0.877771,"redis_calls":10,"redis_duration_s":0.004364,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001723,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090255,"completed_at":"2026-11-07T11:08:07.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023013,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 573.257401] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.827Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ea39a97cfb3f9a6688f53aab","created_at":"2026-11-07T11:08:06.875Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.880Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ea39a97cfb3f9a6688f53aab: start","job_status":"start","queue_duration_s":0.946837,"scheduling_latency_s":0.946867} vm-test-run-nixbot-gitlab> gitlab # [ 573.273639] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.873Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ea39a97cfb3f9a6688f53aab","created_at":"2026-11-07T11:08:06.875Z","trace_propagation_headers":{"sentry-trace":"9b08c996e7354f788c92413354640924-1813cd2583fe4938","baggage":"sentry-trace_id=9b08c996e7354f788c92413354640924,sentry-release=Unknown"},"correlation_id":"01M6TV3CRMXAV887QBT536HZ1V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.880Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ea39a97cfb3f9a6688f53aab: done: 0.046312 sec","job_status":"done","queue_duration_s":0.946837,"scheduling_latency_s":0.946867,"redis_calls":8,"redis_duration_s":0.002075,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046312,"completed_at":"2026-11-07T11:08:07.873Z","load_balancing_strategy":"primary","db_duration_s":0.006697,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 573.316702] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:07.875Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4955b2850eb68a77ad3ee297","created_at":"2026-11-07T11:08:06.976Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.983Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4955b2850eb68a77ad3ee297: start","job_status":"start","queue_duration_s":0.892021,"scheduling_latency_s":0.892052} vm-test-run-nixbot-gitlab> gitlab # [ 573.334185] gitaly[928]: time="2026-11-07T11:08:07.988Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:37.886" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:07.986" grpc.time_ms=1.289 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 573.346913] gitaly[928]: time="2026-11-07T11:08:08.042Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3EZ6P5Z5T4D43GR1EFCN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:38.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:08.040" grpc.time_ms=1.004 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 573.360839] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:08.060Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4955b2850eb68a77ad3ee297","created_at":"2026-11-07T11:08:06.976Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-1f01dcb5857e44a7","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:06.983Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4955b2850eb68a77ad3ee297: done: 0.185084 sec","job_status":"done","queue_duration_s":0.892021,"scheduling_latency_s":0.892052,"gitaly_calls":2,"gitaly_duration_s":0.008588,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.009098,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001366,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001499,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.00034,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002175,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000313,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000681,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140092,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13207792,"duration_s":0.185084,"completed_at":"2026-11-07T11:08:08.060Z","load_balancing_strategy":"primary","db_duration_s":0.038492,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 573.414632] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:08.063Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b4bb6dd63f3c845f5117e2d1","created_at":"2026-11-07T11:08:07.964Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-793ec9c91a1344ce","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:07.967Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b4bb6dd63f3c845f5117e2d1: start","job_status":"start","queue_duration_s":0.096008,"scheduling_latency_s":0.096043} vm-test-run-nixbot-gitlab> gitlab # [ 573.562514] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:08.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b4bb6dd63f3c845f5117e2d1","created_at":"2026-11-07T11:08:07.964Z","trace_propagation_headers":{"sentry-trace":"5f747ab590ee4eb8a3c4bd989024d8b3-793ec9c91a1344ce","baggage":"sentry-trace_id=5f747ab590ee4eb8a3c4bd989024d8b3,sentry-release=Unknown"},"correlation_id":"01M6TV3EZ6P5Z5T4D43GR1EFCN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:07.967Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b4bb6dd63f3c845f5117e2d1: done: 0.222472 sec","job_status":"done","queue_duration_s":0.096008,"scheduling_latency_s":0.096043,"redis_calls":7,"redis_duration_s":0.002393,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.222472,"completed_at":"2026-11-07T11:08:08.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 574.157498] gitaly[928]: time="2026-11-07T11:08:08.880Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3H6BX8ZGQKNR3X9HAJ1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:38.879" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:08.879" grpc.time_ms=0.934 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 574.211953] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 155 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.20 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 575.345429] gitaly[928]: time="2026-11-07T11:08:10.068Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3JBQYF2RVJCCJXK7NFCQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:40.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:10.067" grpc.time_ms=1.097 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 575.388199] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 135 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 576.505455] gitaly[928]: time="2026-11-07T11:08:11.228Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3KGF62S2C4HP64FJPPY7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:41.225" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:11.226" grpc.time_ms=2.325 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 576.560881] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 132 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 577.135170] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:11.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f1213281c30f8b315fe88616","created_at":"2026-11-07T11:08:11.791Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37c4ed027db9fb6e55ba253779814ef0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:08:11.849Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f1213281c30f8b315fe88616: start","job_status":"start","queue_duration_s":0.007855,"scheduling_latency_s":0.0079} vm-test-run-nixbot-gitlab> gitlab # [ 577.304652] nix-daemon[4531]: accepted connection from pid 5802, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 577.324585] nix-daemon[4531]: accepted connection from pid 5761, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 577.465606] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:12.188Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f1213281c30f8b315fe88616","created_at":"2026-11-07T11:08:11.791Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37c4ed027db9fb6e55ba253779814ef0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-07T11:08:11.849Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f1213281c30f8b315fe88616: done: 0.33103 sec","job_status":"done","queue_duration_s":0.007855,"scheduling_latency_s":0.0079,"redis_calls":9,"redis_duration_s":0.028803000000000002,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006588,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006395,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.076,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.33103,"completed_at":"2026-11-07T11:08:12.188Z","load_balancing_strategy":"primary","db_duration_s":0.103541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 577.526102] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:12.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2f5cccb51d7b100a6647a8fd","created_at":"2026-11-07T11:08:11.874Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41e711105097da3df976c267fe882c31","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:11.919Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2f5cccb51d7b100a6647a8fd: start","job_status":"start","queue_duration_s":0.282382,"scheduling_latency_s":0.282432} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 577.694093] gitaly[928]: time="2026-11-07T11:08:12.417Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3MN950Q8NDWHDW9YSP56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:42.415" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:12.416" grpc.time_ms=0.948 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 577.747687] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 141 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 577.921337] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:12.644Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2f5cccb51d7b100a6647a8fd","created_at":"2026-11-07T11:08:11.874Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41e711105097da3df976c267fe882c31","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:11.919Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2f5cccb51d7b100a6647a8fd: done: 0.443017 sec","job_status":"done","queue_duration_s":0.282382,"scheduling_latency_s":0.282432,"redis_calls":9,"redis_duration_s":0.067594,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.027393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.031005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009196,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.074,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.074,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.443017,"completed_at":"2026-11-07T11:08:12.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.106209,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 577.985188] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:12.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9aa50ff3d4d46bd2ea877119","created_at":"2026-11-07T11:08:11.942Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92498632ee9b38ea646273caf905c038","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:11.998Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9aa50ff3d4d46bd2ea877119: start","job_status":"start","queue_duration_s":0.663794,"scheduling_latency_s":0.663845} vm-test-run-nixbot-gitlab> gitlab # [ 578.475331] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9aa50ff3d4d46bd2ea877119","created_at":"2026-11-07T11:08:11.942Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92498632ee9b38ea646273caf905c038","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:11.998Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9aa50ff3d4d46bd2ea877119: done: 0.535567 sec","job_status":"done","queue_duration_s":0.663794,"scheduling_latency_s":0.663845,"redis_calls":11,"redis_duration_s":0.042689,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008906,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.021299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.012484,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.141,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.141,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00387722,"exclusive_lock_hold_duration_s":0.17461848,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.535567,"completed_at":"2026-11-07T11:08:13.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.216239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 578.544510] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"62fea262b9395f98b5f32c49","created_at":"2026-11-07T11:08:12.061Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"672d1ada3f89fd88a691b1cbfe523c9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.234Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-62fea262b9395f98b5f32c49: start","job_status":"start","queue_duration_s":0.976055,"scheduling_latency_s":0.976105} vm-test-run-nixbot-gitlab> gitlab # [ 578.656332] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.380Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 578.732003] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"62fea262b9395f98b5f32c49","created_at":"2026-11-07T11:08:12.061Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"672d1ada3f89fd88a691b1cbfe523c9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.234Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-62fea262b9395f98b5f32c49: done: 0.244408 sec","job_status":"done","queue_duration_s":0.976055,"scheduling_latency_s":0.976105,"redis_calls":9,"redis_duration_s":0.031082000000000002,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00906,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.244408,"completed_at":"2026-11-07T11:08:13.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 578.798219] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.468Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"33e2395118e78ecfddc69bfc","created_at":"2026-11-07T11:08:12.300Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbf76f6583350b11bd4bcd40e253c761","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.372Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-33e2395118e78ecfddc69bfc: start","job_status":"start","queue_duration_s":1.095618,"scheduling_latency_s":1.095668} vm-test-run-nixbot-gitlab> gitlab # [ 578.905725] gitaly[928]: time="2026-11-07T11:08:13.628Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3NV42101T1Z4MX08WG4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:43.627" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:13.627" grpc.time_ms=0.875 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 578.960638] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 143 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.21 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 579.252859] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.975Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"33e2395118e78ecfddc69bfc","created_at":"2026-11-07T11:08:12.300Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbf76f6583350b11bd4bcd40e253c761","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.372Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-33e2395118e78ecfddc69bfc: done: 0.507978 sec","job_status":"done","queue_duration_s":1.095618,"scheduling_latency_s":1.095668,"redis_calls":11,"redis_duration_s":0.038279999999999995,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.009104,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005964,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.507978,"completed_at":"2026-11-07T11:08:13.975Z","load_balancing_strategy":"primary","db_duration_s":0.023288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 579.307243] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:13.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"028e0560a9b83b3d825f566f","created_at":"2026-11-07T11:08:12.401Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc639c1eee5f6b9775e753765e297cb6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.495Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-028e0560a9b83b3d825f566f: start","job_status":"start","queue_duration_s":1.501948,"scheduling_latency_s":1.502} vm-test-run-nixbot-gitlab> gitlab # [ 579.548171] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"028e0560a9b83b3d825f566f","created_at":"2026-11-07T11:08:12.401Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc639c1eee5f6b9775e753765e297cb6","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.495Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-028e0560a9b83b3d825f566f: done: 0.27331 sec","job_status":"done","queue_duration_s":1.501948,"scheduling_latency_s":1.502,"redis_calls":9,"redis_duration_s":0.028609,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008814,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.27331,"completed_at":"2026-11-07T11:08:14.270Z","load_balancing_strategy":"primary","db_duration_s":0.045387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 579.614397] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bbec86c8ec4a2097f900171f","created_at":"2026-11-07T11:08:12.527Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77787836cf24b74901cbdd316042fd69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.567Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bbec86c8ec4a2097f900171f: start","job_status":"start","queue_duration_s":1.715747,"scheduling_latency_s":1.715794} vm-test-run-nixbot-gitlab> gitlab # [ 579.644084] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.367Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 579.684977] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bbec86c8ec4a2097f900171f","created_at":"2026-11-07T11:08:12.527Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77787836cf24b74901cbdd316042fd69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.567Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bbec86c8ec4a2097f900171f: done: 0.125205 sec","job_status":"done","queue_duration_s":1.715747,"scheduling_latency_s":1.715794,"redis_calls":9,"redis_duration_s":0.018208,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005142,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002904,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.125205,"completed_at":"2026-11-07T11:08:14.407Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 579.748908] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ff230389258c77b129056d5d","created_at":"2026-11-07T11:08:12.613Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78e7b90125b7dff36137d2c483f37a08","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:08:12.657Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ff230389258c77b129056d5d: start","job_status":"start","queue_duration_s":1.760877,"scheduling_latency_s":1.760919} vm-test-run-nixbot-gitlab> gitlab # [ 579.775419] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.499Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 579.821863] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ff230389258c77b129056d5d","created_at":"2026-11-07T11:08:12.613Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78e7b90125b7dff36137d2c483f37a08","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-07T11:08:12.657Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ff230389258c77b129056d5d: done: 0.12624 sec","job_status":"done","queue_duration_s":1.760877,"scheduling_latency_s":1.760919,"redis_calls":9,"redis_duration_s":0.023844999999999998,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007116,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.12624,"completed_at":"2026-11-07T11:08:14.544Z","load_balancing_strategy":"primary","db_duration_s":0.003822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 579.888310] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:14.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1a92b2d726ea77817e5f050d","created_at":"2026-11-07T11:08:12.683Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11577e6f0419003be9caa4a6d8c5df5f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.721Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1a92b2d726ea77817e5f050d: start","job_status":"start","queue_duration_s":1.834701,"scheduling_latency_s":1.834745} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 580.096738] gitaly[928]: time="2026-11-07T11:08:14.817Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3Q07TZX6KPCX857HXQAF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:44.815" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:14.816" grpc.time_ms=0.996 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 580.138882] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 134 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 581.259986] gitaly[928]: time="2026-11-07T11:08:15.983Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3R4XHCTEKC9VM3DJWAGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:45.581" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:15.982" grpc.time_ms=0.883 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 581.311742] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 133 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 581.468528] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1a92b2d726ea77817e5f050d","created_at":"2026-11-07T11:08:12.683Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11577e6f0419003be9caa4a6d8c5df5f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.721Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1a92b2d726ea77817e5f050d: done: 1.635109 sec","job_status":"done","queue_duration_s":1.834701,"scheduling_latency_s":1.834745,"redis_calls":564,"redis_duration_s":0.9788629999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005022,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.966706,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.749848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.635109,"completed_at":"2026-11-07T11:08:16.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.538268] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6316450f08a3f621a0853342","created_at":"2026-11-07T11:08:12.748Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d7d192f5b11828503757f102f01e91b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.815Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6316450f08a3f621a0853342: start","job_status":"start","queue_duration_s":3.378515,"scheduling_latency_s":3.378556} vm-test-run-nixbot-gitlab> gitlab # [ 581.557786] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6316450f08a3f621a0853342","created_at":"2026-11-07T11:08:12.748Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d7d192f5b11828503757f102f01e91b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.815Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6316450f08a3f621a0853342: done: 0.046493 sec","job_status":"done","queue_duration_s":3.378515,"scheduling_latency_s":3.378556,"redis_calls":11,"redis_duration_s":0.004085999999999999,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00103678,"exclusive_lock_hold_duration_s":0.00911,"duration_s":0.046493,"completed_at":"2026-11-07T11:08:16.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.603936] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ed9d4a0d569777852067cf6d","created_at":"2026-11-07T11:08:12.851Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e5e7bdf619ce45bcc3d7afe40faa40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:08:12.897Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ed9d4a0d569777852067cf6d: start","job_status":"start","queue_duration_s":3.344836,"scheduling_latency_s":3.344869} vm-test-run-nixbot-gitlab> gitlab # [ 581.617873] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.261Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 581.621741] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ed9d4a0d569777852067cf6d","created_at":"2026-11-07T11:08:12.851Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e5e7bdf619ce45bcc3d7afe40faa40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-07T11:08:12.897Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ed9d4a0d569777852067cf6d: done: 0.029558 sec","job_status":"done","queue_duration_s":3.344836,"scheduling_latency_s":3.344869,"redis_calls":9,"redis_duration_s":0.0020369999999999997,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029558,"completed_at":"2026-11-07T11:08:16.272Z","load_balancing_strategy":"primary","db_duration_s":0.000851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.678024] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1d4bf6aac0c419d752b9ce29","created_at":"2026-11-07T11:08:12.933Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccb405f461aaea57ebd2badc0ea76390","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.970Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1d4bf6aac0c419d752b9ce29: start","job_status":"start","queue_duration_s":3.304211,"scheduling_latency_s":3.304243} vm-test-run-nixbot-gitlab> gitlab # [ 581.694401] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1d4bf6aac0c419d752b9ce29","created_at":"2026-11-07T11:08:12.933Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccb405f461aaea57ebd2badc0ea76390","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:12.970Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1d4bf6aac0c419d752b9ce29: done: 0.040127 sec","job_status":"done","queue_duration_s":3.304211,"scheduling_latency_s":3.304243,"redis_calls":9,"redis_duration_s":0.002373,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000511,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040127,"completed_at":"2026-11-07T11:08:16.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.743986] nix-daemon[4531]: accepted connection from pid 5761, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 581.745204] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8d4933c046aaa3a772cb9ee8","created_at":"2026-11-07T11:08:13.034Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a483c69f5a8287327b4f9ec76df7b26b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.084Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8d4933c046aaa3a772cb9ee8: start","job_status":"start","queue_duration_s":3.232548,"scheduling_latency_s":3.232578} vm-test-run-nixbot-gitlab> gitlab # [ 581.760787] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8d4933c046aaa3a772cb9ee8","created_at":"2026-11-07T11:08:13.034Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a483c69f5a8287327b4f9ec76df7b26b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.084Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8d4933c046aaa3a772cb9ee8: done: 0.04335 sec","job_status":"done","queue_duration_s":3.232548,"scheduling_latency_s":3.232578,"redis_calls":12,"redis_duration_s":0.003513,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001673,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01195122,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04335,"completed_at":"2026-11-07T11:08:16.360Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00499,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 581.810176] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.362Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"02a47acb9dca1739adc43b27","created_at":"2026-11-07T11:08:13.111Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"534ac7cb485d8081c17fcb507a7d14e4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.147Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-02a47acb9dca1739adc43b27: start","job_status":"start","queue_duration_s":3.21523,"scheduling_latency_s":3.215263} vm-test-run-nixbot-gitlab> gitlab # [ 581.825924] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.427Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"02a47acb9dca1739adc43b27","created_at":"2026-11-07T11:08:13.111Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"534ac7cb485d8081c17fcb507a7d14e4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.147Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-02a47acb9dca1739adc43b27: done: 0.064704 sec","job_status":"done","queue_duration_s":3.21523,"scheduling_latency_s":3.215263,"redis_calls":11,"redis_duration_s":0.004146,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000635,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0014838,"exclusive_lock_hold_duration_s":0.01182574,"duration_s":0.064704,"completed_at":"2026-11-07T11:08:16.427Z","load_balancing_strategy":"primary","db_duration_s":0.025686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.870540] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.429Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"de59d8087e48ab61ff236dab","created_at":"2026-11-07T11:08:13.177Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cf45c81316081254afff4cc137a06740","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.213Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-de59d8087e48ab61ff236dab: start","job_status":"start","queue_duration_s":3.216027,"scheduling_latency_s":3.216063} vm-test-run-nixbot-gitlab> gitlab # [ 581.885889] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.463Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"de59d8087e48ab61ff236dab","created_at":"2026-11-07T11:08:13.177Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cf45c81316081254afff4cc137a06740","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.213Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-de59d8087e48ab61ff236dab: done: 0.03404 sec","job_status":"done","queue_duration_s":3.216027,"scheduling_latency_s":3.216063,"redis_calls":9,"redis_duration_s":0.002708,"redis_read_bytes":7,"redis_write_bytes":1108,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000744,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03404,"completed_at":"2026-11-07T11:08:16.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 581.929485] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"70398926d745618042e31e30","created_at":"2026-11-07T11:08:13.235Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8707a08935f983031e097504dfe6aa40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.269Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-70398926d745618042e31e30: start","job_status":"start","queue_duration_s":3.19627,"scheduling_latency_s":3.196301} vm-test-run-nixbot-gitlab> gitlab # [ 581.946509] gitaly[928]: time="2026-11-07T11:08:16.628Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:46.226" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:16.626" grpc.time_ms=1.101 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 581.994015] gitaly[928]: time="2026-11-07T11:08:16.717Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:26.016" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:16.707" grpc.time_ms=10.596 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 582.094160] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 292 vm-test-run-nixbot-gitlab> gitlab # [ 582.137455] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"70398926d745618042e31e30","created_at":"2026-11-07T11:08:13.235Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8707a08935f983031e097504dfe6aa40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.269Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-70398926d745618042e31e30: done: 0.394752 sec","job_status":"done","queue_duration_s":3.19627,"scheduling_latency_s":3.196301,"redis_calls":13,"redis_duration_s":0.005412,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001843,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.319911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00119376,"exclusive_lock_hold_duration_s":0.35300402,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.394752,"completed_at":"2026-11-07T11:08:16.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.090701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.206976] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:16.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e68bdabcb762616ef4407b3c","created_at":"2026-11-07T11:08:13.276Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.317Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e68bdabcb762616ef4407b3c: start","job_status":"start","queue_duration_s":3.545058,"scheduling_latency_s":3.545089} vm-test-run-nixbot-gitlab> gitlab # [ 582.223425] gitaly[928]: time="2026-11-07T11:08:16.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3S1YMP68TS19BVT095E4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:46.532" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:16.933" grpc.time_ms=1.009 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 582.296585] gitaly[928]: time="2026-11-07T11:08:17.019Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3S1YMP68TS19BVT095E4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:27.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.006" grpc.time_ms=13.346 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 582.328164] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e68bdabcb762616ef4407b3c","created_at":"2026-11-07T11:08:13.276Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.317Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e68bdabcb762616ef4407b3c: done: 0.188585 sec","job_status":"done","queue_duration_s":3.545058,"scheduling_latency_s":3.545089,"redis_calls":70,"redis_duration_s":0.024121,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003285,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000751,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.019322,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.188585,"completed_at":"2026-11-07T11:08:17.051Z","load_balancing_strategy":"primary","db_duration_s":0.003723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.413729] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"abe703d4834ba691a68898aa","created_at":"2026-11-07T11:08:13.353Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31b59fcff428ee913f40c5040dd43bf9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.396Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-abe703d4834ba691a68898aa: start","job_status":"start","queue_duration_s":3.657643,"scheduling_latency_s":3.657678} vm-test-run-nixbot-gitlab> gitlab # [ 582.436633] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.074Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 582.440946] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"abe703d4834ba691a68898aa","created_at":"2026-11-07T11:08:13.353Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31b59fcff428ee913f40c5040dd43bf9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.396Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-abe703d4834ba691a68898aa: done: 0.031397 sec","job_status":"done","queue_duration_s":3.657643,"scheduling_latency_s":3.657678,"redis_calls":9,"redis_duration_s":0.002623,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031397,"completed_at":"2026-11-07T11:08:17.085Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.501790] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 284 vm-test-run-nixbot-gitlab> gitlab # [ 582.506488] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3eb9501f31d2ded8dd04241f","created_at":"2026-11-07T11:08:13.435Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6443762681f51eaa573390161687dd6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.480Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3eb9501f31d2ded8dd04241f: start","job_status":"start","queue_duration_s":3.607775,"scheduling_latency_s":3.607824} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.25 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 582.530977] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3eb9501f31d2ded8dd04241f","created_at":"2026-11-07T11:08:13.435Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6443762681f51eaa573390161687dd6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.480Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3eb9501f31d2ded8dd04241f: done: 0.044961 sec","job_status":"done","queue_duration_s":3.607775,"scheduling_latency_s":3.607824,"redis_calls":11,"redis_duration_s":0.004444,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001763,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044961,"completed_at":"2026-11-07T11:08:17.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.598058] gitaly[928]: time="2026-11-07T11:08:17.225Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3SBZWRJBBXRD1TR1MM3E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:47.223" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:17.223" grpc.time_ms=1.176 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 582.618364] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 120 vm-test-run-nixbot-gitlab> gitlab # [ 582.621866] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"720bbb6fc5f3d02b5ba94d3b","created_at":"2026-11-07T11:08:13.571Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"936186c5f960b1e9da98a0de171b4aaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.596Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-720bbb6fc5f3d02b5ba94d3b: start","job_status":"start","queue_duration_s":3.538288,"scheduling_latency_s":3.538326} vm-test-run-nixbot-gitlab> gitlab # [ 582.642941] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.153Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 582.647910] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"720bbb6fc5f3d02b5ba94d3b","created_at":"2026-11-07T11:08:13.571Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"936186c5f960b1e9da98a0de171b4aaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.596Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-720bbb6fc5f3d02b5ba94d3b: done: 0.028438 sec","job_status":"done","queue_duration_s":3.538288,"scheduling_latency_s":3.538326,"redis_calls":9,"redis_duration_s":0.002108,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028438,"completed_at":"2026-11-07T11:08:17.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.712898] gitaly[928]: time="2026-11-07T11:08:17.257Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3SBEGGV4K15CA6J9DXV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:47.252" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:17.252" grpc.time_ms=4.252 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 582.728668] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/df7e7ae3022955d8a907d68546822b663a3d4ff9 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 283 vm-test-run-nixbot-gitlab> gitlab # [ 582.732128] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5bed37ccf601244819b02b41","created_at":"2026-11-07T11:08:13.619Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebae8f6be3285861d8d94f7e9da8e2c8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.674Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5bed37ccf601244819b02b41: start","job_status":"start","queue_duration_s":3.491145,"scheduling_latency_s":3.49118} vm-test-run-nixbot-gitlab> gitlab # [ 582.753871] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5bed37ccf601244819b02b41","created_at":"2026-11-07T11:08:13.619Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebae8f6be3285861d8d94f7e9da8e2c8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.674Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5bed37ccf601244819b02b41: done: 0.03845 sec","job_status":"done","queue_duration_s":3.491145,"scheduling_latency_s":3.49118,"redis_calls":9,"redis_duration_s":0.003817,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03845,"completed_at":"2026-11-07T11:08:17.203Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.814317] gitaly[928]: time="2026-11-07T11:08:17.330Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3SBEGGV4K15CA6J9DXV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-07T11:08:27.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.320" grpc.time_ms=9.688 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 582.836086] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6ce5fbf8971c3a0ab144968a","created_at":"2026-11-07T11:08:13.703Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f440e535d5dd8662a94e61b732d928d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.757Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6ce5fbf8971c3a0ab144968a: start","job_status":"start","queue_duration_s":3.452393,"scheduling_latency_s":3.452431} vm-test-run-nixbot-gitlab> gitlab # [ 582.859694] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6ce5fbf8971c3a0ab144968a","created_at":"2026-11-07T11:08:13.703Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f440e535d5dd8662a94e61b732d928d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:13.757Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6ce5fbf8971c3a0ab144968a: done: 0.053389 sec","job_status":"done","queue_duration_s":3.452393,"scheduling_latency_s":3.452431,"redis_calls":11,"redis_duration_s":0.005137,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000591,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002656,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00103878,"exclusive_lock_hold_duration_s":0.0065682,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.053389,"completed_at":"2026-11-07T11:08:17.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 582.919457] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3915997217d1c1369bdeaecd","created_at":"2026-11-07T11:08:13.963Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa3d7c392336ea41a7d81bfb16f7ffbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.012Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3915997217d1c1369bdeaecd: start","job_status":"start","queue_duration_s":3.253922,"scheduling_latency_s":3.25396} vm-test-run-nixbot-gitlab> gitlab # [ 582.940747] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.301Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3915997217d1c1369bdeaecd","created_at":"2026-11-07T11:08:13.963Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa3d7c392336ea41a7d81bfb16f7ffbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.012Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3915997217d1c1369bdeaecd: done: 0.034909 sec","job_status":"done","queue_duration_s":3.253922,"scheduling_latency_s":3.25396,"redis_calls":9,"redis_duration_s":0.002733,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000591,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000716,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034909,"completed_at":"2026-11-07T11:08:17.300Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.011598] gitaly[928]: time="2026-11-07T11:08:17.656Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:17.655" grpc.time_ms=0.71 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.023619] gitaly[928]: time="2026-11-07T11:08:17.692Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:17.691" grpc.time_ms=0.592 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.035572] nixbot[4361]: {"ts": "2026-11-07T11:08:17.705Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "df7e7ae3022955d8a907d68546822b663a3d4ff9", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} vm-test-run-nixbot-gitlab> gitlab # [ 583.039898] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d0beef9598edf40a3ecd4e30","created_at":"2026-11-07T11:08:14.037Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf0af0b4773131b3d5ab6793d5700b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.080Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d0beef9598edf40a3ecd4e30: start","job_status":"start","queue_duration_s":3.222695,"scheduling_latency_s":3.222728} vm-test-run-nixbot-gitlab> gitlab # [ 583.057183] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.397Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d0beef9598edf40a3ecd4e30","created_at":"2026-11-07T11:08:14.037Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf0af0b4773131b3d5ab6793d5700b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.080Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d0beef9598edf40a3ecd4e30: done: 0.094521 sec","job_status":"done","queue_duration_s":3.222695,"scheduling_latency_s":3.222728,"redis_calls":13,"redis_duration_s":0.0043560000000000005,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001252,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00102274,"exclusive_lock_hold_duration_s":0.0483079,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.094521,"completed_at":"2026-11-07T11:08:17.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.125246] gitaly[928]: time="2026-11-07T11:08:17.725Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-07T11:08:17.724" grpc.time_ms=0.589 pid=928 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.136389] gitaly[928]: time="2026-11-07T11:08:17.728Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" operations="[repack_refs]" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.148864] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6285da1a5c65b37686e8716b","created_at":"2026-11-07T11:08:14.112Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde51e2f15a259e2dfc47ac4677084cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.162Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6285da1a5c65b37686e8716b: start","job_status":"start","queue_duration_s":3.237552,"scheduling_latency_s":3.237592} vm-test-run-nixbot-gitlab> gitlab # [ 583.168811] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6285da1a5c65b37686e8716b","created_at":"2026-11-07T11:08:14.112Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-d02e891629394046","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde51e2f15a259e2dfc47ac4677084cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:14.162Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6285da1a5c65b37686e8716b: done: 0.066072 sec","job_status":"done","queue_duration_s":3.237552,"scheduling_latency_s":3.237592,"redis_calls":18,"redis_duration_s":0.005651,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001327,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00049,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00272,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066072,"completed_at":"2026-11-07T11:08:17.466Z","load_balancing_strategy":"primary","db_duration_s":0.003824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.225740] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 94 vm-test-run-nixbot-gitlab> gitlab # [ 583.228224] gitaly[928]: time="2026-11-07T11:08:17.731Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" pid=928 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-07 11:05:58.38572826 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.246147] gitaly[928]: time="2026-11-07T11:08:17.736Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V diskcache=c551ce0e-2efe-4b06-8fca-f01d3494c462 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.261392] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.468Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"9b30688237106dffbef55962","created_at":"2026-11-07T11:08:16.787Z","trace_propagation_headers":{"sentry-trace":"580feb7edca143b486963581856ba1cc-ab1ad53fcf8a4b8d","baggage":"sentry-trace_id=580feb7edca143b486963581856ba1cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3RRD7X1TYWBRMPQTAF3V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:16.793Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9b30688237106dffbef55962: start","job_status":"start","queue_duration_s":0.67463,"scheduling_latency_s":0.674669} vm-test-run-nixbot-gitlab> gitlab # [ 583.282287] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.948Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"9b30688237106dffbef55962","created_at":"2026-11-07T11:08:16.787Z","trace_propagation_headers":{"sentry-trace":"580feb7edca143b486963581856ba1cc-ab1ad53fcf8a4b8d","baggage":"sentry-trace_id=580feb7edca143b486963581856ba1cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3RRD7X1TYWBRMPQTAF3V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:16.793Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":4230,"job_status":"rescheduled","message":"PipelineProcessWorker JID-9b30688237106dffbef55962: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 583.302374] gitaly[928]: time="2026-11-07T11:08:17.736Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=98 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7106 command.oublock=0 command.real_time_ms=106 command.spawn_token_fork_ms=1 command.system_time_ms=46 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" grpc.time_ms=115.475 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.318252] gitlab-workhorse[830]: time="2026-11-07T11:08:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 583.320205] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 123 vm-test-run-nixbot-gitlab> gitlab # [ 583.322289] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.962Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"9b30688237106dffbef55962","created_at":"2026-11-07T11:08:16.787Z","trace_propagation_headers":{"sentry-trace":"580feb7edca143b486963581856ba1cc-ab1ad53fcf8a4b8d","baggage":"sentry-trace_id=580feb7edca143b486963581856ba1cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6TV3RRD7X1TYWBRMPQTAF3V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-07T11:08:16.793Z","job_size_bytes":3,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9b30688237106dffbef55962: done: 0.494148 sec","job_status":"done","queue_duration_s":0.67463,"scheduling_latency_s":0.674669,"gitaly_calls":3,"gitaly_duration_s":0.133587,"redis_calls":76,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.023971,"redis_read_bytes":168,"redis_write_bytes":21330,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001318,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.003554,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.005452,"redis_queues_read_bytes":37,"redis_queues_write_bytes":14468,"redis_queues_metadata_calls":20,"redis_queues_metadata_duration_s":0.009256,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3301,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000303,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00052,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00225,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.017,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.021,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.04,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257535,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.41268788,"duration_s":0.494148,"completed_at":"2026-11-07T11:08:17.962Z","load_balancing_strategy":"primary","db_duration_s":0.08762,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 583.378868] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:17.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a3a4dee272f6229ff28b662","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a3a4dee272f6229ff28b662: start","job_status":"start","queue_duration_s":0.926638,"scheduling_latency_s":0.926677} vm-test-run-nixbot-gitlab> gitlab # [ 583.392573] gitaly[928]: time="2026-11-07T11:08:17.776Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" pid=928 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.407458] gitaly[928]: time="2026-11-07T11:08:17.776Z" level=info msg="optimized repository" clean_stale_data_time=44.2805ms clean_work_trees_time="138.26µs" commit_graph_time=740ns correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.92µs" pid=928 prune_time=260ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.92µs" storage_name=default system=housekeeping total_time=44.45502ms vm-test-run-nixbot-gitlab> gitlab # [ 583.413933] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a3a4dee272f6229ff28b662","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a3a4dee272f6229ff28b662: done: 0.048268 sec","job_status":"done","queue_duration_s":0.926638,"scheduling_latency_s":0.926677,"redis_calls":15,"redis_duration_s":0.006337,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048268,"completed_at":"2026-11-07T11:08:18.013Z","load_balancing_strategy":"primary","db_duration_s":0.002282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.461893] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16af387797c86f3d57ed14f2","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16af387797c86f3d57ed14f2: start","job_status":"start","queue_duration_s":0.978115,"scheduling_latency_s":0.978159} vm-test-run-nixbot-gitlab> gitlab # [ 583.478889] gitaly[928]: time="2026-11-07T11:08:17.776Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-07T11:08:47.520" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-07T11:08:17.621" pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.492501] gitaly[928]: time="2026-11-07T11:08:17.853Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3SYY8SGJ4SMRRGHZ0XNB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:17.842" grpc.time_ms=10.966 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.511995] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 111 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses vm-test-run-nixbot-gitlab> gitlab # [ 583.515654] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16af387797c86f3d57ed14f2","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16af387797c86f3d57ed14f2: done: 0.039925 sec","job_status":"done","queue_duration_s":0.978115,"scheduling_latency_s":0.978159,"redis_calls":13,"redis_duration_s":0.004431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039925,"completed_at":"2026-11-07T11:08:18.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.573657] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56eddb57640fba9aa206500","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56eddb57640fba9aa206500: start","job_status":"start","queue_duration_s":1.020912,"scheduling_latency_s":1.020982} vm-test-run-nixbot-gitlab> gitlab # [ 583.592249] gitaly[928]: time="2026-11-07T11:08:17.868Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-07T11:08:47.766" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:17.867" grpc.time_ms=0.999 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.609054] gitlab-workhorse[830]: time="2026-11-07T11:08:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 583.611982] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 118 vm-test-run-nixbot-gitlab> gitlab # [ 583.615200] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56eddb57640fba9aa206500","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56eddb57640fba9aa206500: done: 0.046197 sec","job_status":"done","queue_duration_s":1.020912,"scheduling_latency_s":1.020982,"redis_calls":13,"redis_duration_s":0.015624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.046197,"completed_at":"2026-11-07T11:08:18.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 583.672957] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5378c5690767a41c3a9f79ec","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5378c5690767a41c3a9f79ec: start","job_status":"start","queue_duration_s":1.073116,"scheduling_latency_s":1.073164} vm-test-run-nixbot-gitlab> gitlab # [ 583.689231] gitaly[928]: time="2026-11-07T11:08:17.927Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3RRD7X1TYWBRMPQTAF3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-07T11:08:47.826" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-07T11:08:17.926" grpc.time_ms=0.686 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 583.708299] gitaly[928]: time="2026-11-07T11:08:17.979Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3T20PMSB5DZ5XKQWP7XC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:17.963" pid=928 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.722067] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5378c5690767a41c3a9f79ec","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5378c5690767a41c3a9f79ec: done: 0.040577 sec","job_status":"done","queue_duration_s":1.073116,"scheduling_latency_s":1.073164,"redis_calls":13,"redis_duration_s":0.005996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003377,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040577,"completed_at":"2026-11-07T11:08:18.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses, in 0.21 seconds) vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 10.99 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 783.55 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 583.766933] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"618f25631fcbbc15150f3ce4","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-618f25631fcbbc15150f3ce4: start","job_status":"start","queue_duration_s":1.116073,"scheduling_latency_s":1.116112} vm-test-run-nixbot-gitlab> gitlab # [ 583.781689] gitaly[928]: time="2026-11-07T11:08:17.979Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M6TV3T20PMSB5DZ5XKQWP7XC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-07T11:08:17.963" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=928 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 583.796482] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/11/07:11:08:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/df7e7ae3022955d8a907d68546822b663a3d4ff9/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 145 vm-test-run-nixbot-gitlab> test script finished in 783.58s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 46) vm-test-run-nixbot-gitlab> gitlab # [ 583.799239] sidekiq-cluster[4230]: {"severity":"INFO","time":"2026-11-07T11:08:18.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"618f25631fcbbc15150f3ce4","created_at":"2026-11-07T11:08:16.892Z","trace_propagation_headers":{"sentry-trace":"16770be1d73e473cbed3749ee3d4ce16-5423bb9c8e784f3a","baggage":"sentry-trace_id=16770be1d73e473cbed3749ee3d4ce16,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e8d2a129da0eb5a1fc584e4ec1cc9645","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-07T11:08:17.038Z","job_size_bytes":2,"pid":4230,"sidekiq_tid":"m67i","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-618f25631fcbqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) vm-test-run-nixbot-gitlab> gitlab # b vm-test-run-nixbot-gitlab> (finished: cleanup, in 4.79 seconds) vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-07T11:06:11.954Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-07T11:08:06.490819Z', 'started_at': '2026-11-07T11:08:07.031919Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': '2026-11-07T11:08:16.755Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-11-07T11:08:17.069Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> [{'id': 5, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-11-07T11:08:06.957Z', 'started_at': None, 'finished_at': '2026-11-07T11:08:16.755Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'df7e7ae3022955d8a907d68546822b663a3d4ff9', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-11-07T11:08:17.069Z', 'started_at': None, 'finished_at': '2026-11-07T11:08:17.366Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=checks.aarch64-linux.nixbot-gitlab