1these 13 derivations will be built:2 /nix/store/iqaxdcr6n31iagwnh4kppg3kmyfs9g07-nixbot-effects.drv3 /nix/store/d1052lzh00wz8313rm0c5miv60wqsmq4-unit-nixbot.service.drv4 /nix/store/karbpidmz3wh1140nnzngxg7grdxggk0-system-units.drv5 /nix/store/g308gr5mrbl0hm6ghy1q2hc19l23smax-etc.drv6 /nix/store/rjz0zypzcq6l8i7d14mpspjzvr98l70y-activate.drv7 /nix/store/dv2law380jpv1i67aaypywl0rvm1wlv1-nixos-system-gitlab-test.drv8 /nix/store/bk74p8kq64dqh143nnrhcnx95l2rnkdv-closure-info.drv9 /nix/store/a3kknsyjxg6f0p8hzwyifrnsi6nfl63p-closure-info.drv10 /nix/store/shiz82csmjq8lrvpnm4gr8b7ppbwhp7j-run-nixos-vm.drv11 /nix/store/dps2fvkp6jzwn7y57sc2j0n0vh0k1yqb-nixos-vm.drv12 /nix/store/0ym0sdyddf08wf1vzgzvc36vr6pjdsg5-driverConfiguration.json.drv13 /nix/store/7jkrddhhfnb91mxyd8yrlsbgzmn4siwq-nixos-test-driver-nixbot-gitlab.drv14 /nix/store/5c25m8m23774cdx4vm212wccv7bkqdi1-vm-test-run-nixbot-gitlab.drv15building '/nix/store/iqaxdcr6n31iagwnh4kppg3kmyfs9g07-nixbot-effects.drv' on 'ssh-ng://nix@eliza'16building '/nix/store/iqaxdcr6n31iagwnh4kppg3kmyfs9g07-nixbot-effects.drv'17nixbot-effects> Sourcing python-remove-tests-dir-hook18nixbot-effects> Sourcing python-catch-conflicts-hook.sh19nixbot-effects> Sourcing python-remove-bin-bytecode-hook.sh20nixbot-effects> Sourcing pypa-build-hook21nixbot-effects> Using pypaBuildPhase22nixbot-effects> Sourcing python-runtime-deps-check-hook23nixbot-effects> Using pythonRuntimeDepsCheckHook24nixbot-effects> Sourcing pypa-install-hook25nixbot-effects> Using pypaInstallPhase26nixbot-effects> Sourcing python-imports-check-hook.sh27nixbot-effects> Using pythonImportsCheckPhase28nixbot-effects> Sourcing python-namespaces-hook29nixbot-effects> Sourcing python-catch-conflicts-hook.sh30nixbot-effects> Running phase: unpackPhase31nixbot-effects> unpacking source archive /nix/store/k00dadfbc7qlmhc9khnzd3z20z7ij5bk-nixbot_effects32nixbot-effects> source root is nixbot_effects33nixbot-effects> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot_effects/pyproject.toml"34nixbot-effects> Running phase: patchPhase35nixbot-effects> Running phase: updateAutotoolsGnuConfigScriptsPhase36nixbot-effects> Running phase: configurePhase37nixbot-effects> no configure script, doing nothing38nixbot-effects> Running phase: buildPhase39nixbot-effects> Executing pypaBuildPhase40nixbot-effects> Creating a wheel...41nixbot-effects> pypa build flags: --no-isolation --outdir dist/ --wheel42nixbot-effects> * Getting build dependencies for wheel...43nixbot-effects> * Building wheel...44nixbot-effects> Successfully built nixbot_effects-0.0.1-py3-none-any.whl45nixbot-effects> Finished creating a wheel...46nixbot-effects> Finished executing pypaBuildPhase47nixbot-effects> Running phase: pythonRuntimeDepsCheckHook48nixbot-effects> Executing pythonRuntimeDepsCheck49nixbot-effects> Checking runtime dependencies for nixbot_effects-0.0.1-py3-none-any.whl50nixbot-effects> Finished executing pythonRuntimeDepsCheck51nixbot-effects> Running phase: installPhase52nixbot-effects> Executing pypaInstallPhase53nixbot-effects> Successfully installed nixbot_effects-0.0.1-py3-none-any.whl54nixbot-effects> Finished executing pypaInstallPhase55nixbot-effects> Running phase: pythonOutputDistPhase56nixbot-effects> Executing pythonOutputDistPhase57nixbot-effects> Finished executing pythonOutputDistPhase58nixbot-effects> Running phase: fixupPhase59nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects60nixbot-effects> checking for references to /build/ in /nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects...61nixbot-effects> patching script interpreter paths in /nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects62nixbot-effects> stripping (with command strip and flags -S -p) in /nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects/lib /nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects/bin63nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/36f9nv0pcwl9g372v2n8j7jqzgjddcn3-nixbot-effects-dist64nixbot-effects> checking for references to /build/ in /nix/store/36f9nv0pcwl9g372v2n8j7jqzgjddcn3-nixbot-effects-dist...65nixbot-effects> patching script interpreter paths in /nix/store/36f9nv0pcwl9g372v2n8j7jqzgjddcn3-nixbot-effects-dist66nixbot-effects> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.1367nixbot-effects> wrapping `/nix/store/wbgpy26yh7yn990w7rpwa35199xvs8ip-nixbot-effects/bin/nixbot-effects'...68nixbot-effects> Executing pythonRemoveTestsDir69nixbot-effects> Finished executing pythonRemoveTestsDir70nixbot-effects> Running phase: installCheckPhase71nixbot-effects> no Makefile or custom installCheckPhase, doing nothing72nixbot-effects> Running phase: pythonCatchConflictsPhase73nixbot-effects> Running phase: pythonRemoveBinBytecodePhase74nixbot-effects> Running phase: pythonImportsCheckPhase75nixbot-effects> Executing pythonImportsCheckPhase76building '/nix/store/d1052lzh00wz8313rm0c5miv60wqsmq4-unit-nixbot.service.drv' on 'ssh-ng://nix@eliza'77building '/nix/store/d1052lzh00wz8313rm0c5miv60wqsmq4-unit-nixbot.service.drv'78building '/nix/store/karbpidmz3wh1140nnzngxg7grdxggk0-system-units.drv' on 'ssh-ng://nix@eliza'79building '/nix/store/karbpidmz3wh1140nnzngxg7grdxggk0-system-units.drv'80building '/nix/store/g308gr5mrbl0hm6ghy1q2hc19l23smax-etc.drv' on 'ssh-ng://nix@eliza'81building '/nix/store/g308gr5mrbl0hm6ghy1q2hc19l23smax-etc.drv'82building '/nix/store/rjz0zypzcq6l8i7d14mpspjzvr98l70y-activate.drv' on 'ssh-ng://nix@eliza'83building '/nix/store/rjz0zypzcq6l8i7d14mpspjzvr98l70y-activate.drv'84building '/nix/store/dv2law380jpv1i67aaypywl0rvm1wlv1-nixos-system-gitlab-test.drv' on 'ssh-ng://nix@eliza'85building '/nix/store/dv2law380jpv1i67aaypywl0rvm1wlv1-nixos-system-gitlab-test.drv'86nixos-system-gitlab-test> structuredAttrs is enabled87building '/nix/store/bk74p8kq64dqh143nnrhcnx95l2rnkdv-closure-info.drv' on 'ssh-ng://nix@eliza'88building '/nix/store/bk74p8kq64dqh143nnrhcnx95l2rnkdv-closure-info.drv'89closure-info> structuredAttrs is enabled90building '/nix/store/a3kknsyjxg6f0p8hzwyifrnsi6nfl63p-closure-info.drv' on 'ssh-ng://nix@eliza'91building '/nix/store/a3kknsyjxg6f0p8hzwyifrnsi6nfl63p-closure-info.drv'92closure-info> structuredAttrs is enabled93building '/nix/store/shiz82csmjq8lrvpnm4gr8b7ppbwhp7j-run-nixos-vm.drv' on 'ssh-ng://nix@eliza'94building '/nix/store/shiz82csmjq8lrvpnm4gr8b7ppbwhp7j-run-nixos-vm.drv'95building '/nix/store/dps2fvkp6jzwn7y57sc2j0n0vh0k1yqb-nixos-vm.drv' on 'ssh-ng://nix@eliza'96building '/nix/store/dps2fvkp6jzwn7y57sc2j0n0vh0k1yqb-nixos-vm.drv'97building '/nix/store/0ym0sdyddf08wf1vzgzvc36vr6pjdsg5-driverConfiguration.json.drv' on 'ssh-ng://nix@eliza'98building '/nix/store/0ym0sdyddf08wf1vzgzvc36vr6pjdsg5-driverConfiguration.json.drv'99driverConfiguration.json> structuredAttrs is enabled100building '/nix/store/7jkrddhhfnb91mxyd8yrlsbgzmn4siwq-nixos-test-driver-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'101building '/nix/store/7jkrddhhfnb91mxyd8yrlsbgzmn4siwq-nixos-test-driver-nixbot-gitlab.drv'102nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)103nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck104nixos-test-driver-nixbot-gitlab> All checks passed!105nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)106nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint107nixos-test-driver-nixbot-gitlab> All checks passed!108building '/nix/store/5c25m8m23774cdx4vm212wccv7bkqdi1-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'109building '/nix/store/5c25m8m23774cdx4vm212wccv7bkqdi1-vm-test-run-nixbot-gitlab.drv'110vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state111vm-test-run-nixbot-gitlab> start all VLans112vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)113vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds114vm-test-run-nixbot-gitlab> run the VM test script115vm-test-run-nixbot-gitlab> additionally exposed symbols:116vm-test-run-nixbot-gitlab> gitlab,117vm-test-run-nixbot-gitlab> vlan1,118vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh119vm-test-run-nixbot-gitlab> start all VMs120vm-test-run-nixbot-gitlab> gitlab: starting vm121vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...122vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)123vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.LIe05HW8CH', fmt=raw size=1073741824124vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)125vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done126vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes127vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 91c97e3c-8f24-4d1f-ae7a-ef10c3ffb8f2128vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:129vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done132vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done133vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done134vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.137vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...138vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.139vm-test-run-nixbot-gitlab> (finished: start all VMs, in 65.73 seconds)140vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service141vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting142vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]143vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun 1 15:51:08 UTC 2026144vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] KASLR enabled145vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] random: crng init done146vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Machine model: linux,dummy-virt147vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] efi: UEFI not found.148vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT149vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]150vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]151vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Zone ranges:152vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]153vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMA32 empty154vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]155vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Device empty156vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Movable zone start for each node157vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Early memory node ranges158vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]159vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]160vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000161vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: probing for conduit method from DT.162vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.163vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs164vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: Trusted OS migration not required165vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] psci: SMC Calling Convention v1.1166vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)167vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296168vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Detected PIPT I-cache on CPU0169vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)170vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface171vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-v4172vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: Spectre-BHB173vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38174vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23175vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] alternatives: applying boot alternatives176vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/99x979qwsg5j0ap0mklhjcwk21as8gkv-nixos-system-gitlab-test/init regInfo=/nix/store/r7rc1jab0qvpb94kp61477cznpqg1yrf-closure-info/registration console=ttyAMA0,115200n8 console=tty0177vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/r7rc1jab0qvpb94kp61477cznpqg1yrf-closure-info/registration", will be passed to user space.178vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes179vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)180vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)181vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: area num 4.182vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)183vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Fallback order for Node 0: 0184vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864185vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Policy zone: Normal186vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off187vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1188vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] allocated 12582912 bytes of page_ext189vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocating 74648 entries in 292 pages190vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ftrace: allocated 292 pages with 3 groups191vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.192vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU event tracing is enabled.193vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.194vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.195vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.196vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.197vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.198vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4199vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.200vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.201vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.202vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0203vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 256 SPIs implemented204vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented205vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq206vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI207vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0208vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000209vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]210vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)211vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)212vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000213vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000214vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.215vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).216vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns217vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns218vm-test-run-nixbot-gitlab> gitlab # [ 0.000026] arm-pv: using stolen time PV219vm-test-run-nixbot-gitlab> gitlab # [ 0.000380] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)220vm-test-run-nixbot-gitlab> gitlab # [ 0.000532] Console: colour dummy device 80x25221vm-test-run-nixbot-gitlab> gitlab # [ 0.000539] printk: legacy console [tty0] enabled222vm-test-run-nixbot-gitlab> gitlab # [ 0.000710] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)223vm-test-run-nixbot-gitlab> gitlab # [ 0.000716] pid_max: default: 32768 minimum: 301224vm-test-run-nixbot-gitlab> gitlab # [ 0.000791] LSM: initializing lsm=capability,landlock,yama,bpf,ima225vm-test-run-nixbot-gitlab> gitlab # [ 0.000932] landlock: Up and running.226vm-test-run-nixbot-gitlab> gitlab # [ 0.000935] Yama: becoming mindful.227vm-test-run-nixbot-gitlab> gitlab # [ 0.001333] LSM support for eBPF active228vm-test-run-nixbot-gitlab> gitlab # [ 0.001541] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)229vm-test-run-nixbot-gitlab> gitlab # [ 0.001649] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)230vm-test-run-nixbot-gitlab> gitlab # [ 0.002696] cacheinfo: Unable to detect cache hierarchy for CPU 0231vm-test-run-nixbot-gitlab> gitlab # [ 0.003355] rcu: Hierarchical SRCU implementation.232vm-test-run-nixbot-gitlab> gitlab # [ 0.003358] rcu: Max phase no-delay instances is 1000.233vm-test-run-nixbot-gitlab> gitlab # [ 0.003510] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level234vm-test-run-nixbot-gitlab> gitlab # [ 0.004576] fsl-mc MSI: its@8080000 domain created235vm-test-run-nixbot-gitlab> gitlab # [ 0.004668] EFI services will not be available.236vm-test-run-nixbot-gitlab> gitlab # [ 0.004855] smp: Bringing up secondary CPUs ...237vm-test-run-nixbot-gitlab> gitlab # [ 0.005497] Detected PIPT I-cache on CPU1238vm-test-run-nixbot-gitlab> gitlab # [ 0.005596] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000239vm-test-run-nixbot-gitlab> gitlab # [ 0.005726] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000240vm-test-run-nixbot-gitlab> gitlab # [ 0.005853] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]241vm-test-run-nixbot-gitlab> gitlab # [ 0.006906] Detected PIPT I-cache on CPU2242vm-test-run-nixbot-gitlab> gitlab # [ 0.007115] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000243vm-test-run-nixbot-gitlab> gitlab # [ 0.007246] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000244vm-test-run-nixbot-gitlab> gitlab # [ 0.007376] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]245vm-test-run-nixbot-gitlab> gitlab # [ 0.008378] Detected PIPT I-cache on CPU3246vm-test-run-nixbot-gitlab> gitlab # [ 0.008496] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000247vm-test-run-nixbot-gitlab> gitlab # [ 0.008627] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000248vm-test-run-nixbot-gitlab> gitlab # [ 0.008756] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]249vm-test-run-nixbot-gitlab> gitlab # [ 0.009289] smp: Brought up 1 node, 4 CPUs250vm-test-run-nixbot-gitlab> gitlab # [ 0.009314] SMP: Total of 4 processors activated.251vm-test-run-nixbot-gitlab> gitlab # [ 0.009317] CPU: All CPU(s) started at EL1252vm-test-run-nixbot-gitlab> gitlab # [ 0.009325] CPU features: detected: Branch Target Identification253vm-test-run-nixbot-gitlab> gitlab # [ 0.009329] CPU features: detected: ARMv8.4 Translation Table Level254vm-test-run-nixbot-gitlab> gitlab # [ 0.009332] CPU features: detected: Instruction cache invalidation not required for I/D coherence255vm-test-run-nixbot-gitlab> gitlab # [ 0.009335] CPU features: detected: Data cache clean to the PoU not required for I/D coherence256vm-test-run-nixbot-gitlab> gitlab # [ 0.009339] CPU features: detected: Common not Private translations257vm-test-run-nixbot-gitlab> gitlab # [ 0.009341] CPU features: detected: CRC32 instructions258vm-test-run-nixbot-gitlab> gitlab # [ 0.009344] CPU features: detected: Data cache clean to Point of Deep Persistence259vm-test-run-nixbot-gitlab> gitlab # [ 0.009347] CPU features: detected: Data cache clean to Point of Persistence260vm-test-run-nixbot-gitlab> gitlab # [ 0.009350] CPU features: detected: Data independent timing control (DIT)261vm-test-run-nixbot-gitlab> gitlab # [ 0.009353] CPU features: detected: E0PD262vm-test-run-nixbot-gitlab> gitlab # [ 0.009355] CPU features: detected: Enhanced Counter Virtualization263vm-test-run-nixbot-gitlab> gitlab # [ 0.009358] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)264vm-test-run-nixbot-gitlab> gitlab # [ 0.009361] CPU features: detected: Enhanced Virtualization Traps265vm-test-run-nixbot-gitlab> gitlab # [ 0.009363] CPU features: detected: Fine Grained Traps266vm-test-run-nixbot-gitlab> gitlab # [ 0.009366] CPU features: detected: Generic authentication (architected QARMA5 algorithm)267vm-test-run-nixbot-gitlab> gitlab # [ 0.009370] CPU features: detected: RCpc load-acquire (LDAPR)268vm-test-run-nixbot-gitlab> gitlab # [ 0.009373] CPU features: detected: LSE atomic instructions269vm-test-run-nixbot-gitlab> gitlab # [ 0.009376] CPU features: detected: Privileged Access Never270vm-test-run-nixbot-gitlab> gitlab # [ 0.009378] CPU features: detected: PMUv3271vm-test-run-nixbot-gitlab> gitlab # [ 0.009381] CPU features: detected: RAS Extension Support272vm-test-run-nixbot-gitlab> gitlab # [ 0.009383] CPU features: detected: RASv1p1 Extension Support273vm-test-run-nixbot-gitlab> gitlab # [ 0.009386] CPU features: detected: Random Number Generator274vm-test-run-nixbot-gitlab> gitlab # [ 0.009388] CPU features: detected: Speculation barrier (SB)275vm-test-run-nixbot-gitlab> gitlab # [ 0.009391] CPU features: detected: Stage-2 Force Write-Back276vm-test-run-nixbot-gitlab> gitlab # [ 0.009393] CPU features: detected: TLB range maintenance instructions277vm-test-run-nixbot-gitlab> gitlab # [ 0.009397] CPU features: detected: Speculative Store Bypassing Safe (SSBS)278vm-test-run-nixbot-gitlab> gitlab # [ 0.009527] alternatives: applying system-wide alternatives279vm-test-run-nixbot-gitlab> gitlab # [ 0.012335] CPU features: detected: BBM Level 2 without TLB conflict abort280vm-test-run-nixbot-gitlab> gitlab # [ 0.012590] Memory: 5965832K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved)281vm-test-run-nixbot-gitlab> gitlab # [ 0.014001] devtmpfs: initialized282vm-test-run-nixbot-gitlab> gitlab # [ 0.017899] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns283vm-test-run-nixbot-gitlab> gitlab # [ 0.017944] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)284vm-test-run-nixbot-gitlab> gitlab # [ 0.018012] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).285vm-test-run-nixbot-gitlab> gitlab # [ 0.018184] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL286vm-test-run-nixbot-gitlab> gitlab # [ 0.018188] 0 pages in range for non-PLT usage287vm-test-run-nixbot-gitlab> gitlab # [ 0.018189] 508336 pages in range for PLT usage288vm-test-run-nixbot-gitlab> gitlab # [ 0.018289] pinctrl core: initialized pinctrl subsystem289vm-test-run-nixbot-gitlab> gitlab # [ 0.019098] DMI not present or invalid.290vm-test-run-nixbot-gitlab> gitlab # [ 0.022261] NET: Registered PF_NETLINK/PF_ROUTE protocol family291vm-test-run-nixbot-gitlab> gitlab # [ 0.024956] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations292vm-test-run-nixbot-gitlab> gitlab # [ 0.025224] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations293vm-test-run-nixbot-gitlab> gitlab # [ 0.027078] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations294vm-test-run-nixbot-gitlab> gitlab # [ 0.027100] audit: initializing netlink subsys (disabled)295vm-test-run-nixbot-gitlab> gitlab # [ 0.027389] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1296vm-test-run-nixbot-gitlab> gitlab # [ 0.027943] thermal_sys: Registered thermal governor 'fair_share'297vm-test-run-nixbot-gitlab> gitlab # [ 0.027946] thermal_sys: Registered thermal governor 'bang_bang'298vm-test-run-nixbot-gitlab> gitlab # [ 0.027950] thermal_sys: Registered thermal governor 'step_wise'299vm-test-run-nixbot-gitlab> gitlab # [ 0.027952] thermal_sys: Registered thermal governor 'user_space'300vm-test-run-nixbot-gitlab> gitlab # [ 0.027955] thermal_sys: Registered thermal governor 'power_allocator'301vm-test-run-nixbot-gitlab> gitlab # [ 0.028039] cpuidle: using governor ladder302vm-test-run-nixbot-gitlab> gitlab # [ 0.028109] cpuidle: using governor menu303vm-test-run-nixbot-gitlab> gitlab # [ 0.028309] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.304vm-test-run-nixbot-gitlab> gitlab # [ 0.028516] ASID allocator initialised with 65536 entries305vm-test-run-nixbot-gitlab> gitlab # [ 0.029694] Serial: AMBA PL011 UART driver306vm-test-run-nixbot-gitlab> gitlab # [ 0.034795] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1307vm-test-run-nixbot-gitlab> gitlab # [ 0.035103] printk: console [ttyAMA0] enabled308vm-test-run-nixbot-gitlab> gitlab # [ 0.047479] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages309vm-test-run-nixbot-gitlab> gitlab # [ 0.047498] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page310vm-test-run-nixbot-gitlab> gitlab # [ 0.047507] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages311vm-test-run-nixbot-gitlab> gitlab # [ 0.047515] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page312vm-test-run-nixbot-gitlab> gitlab # [ 0.047523] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages313vm-test-run-nixbot-gitlab> gitlab # [ 0.047531] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page314vm-test-run-nixbot-gitlab> gitlab # [ 0.047539] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages315vm-test-run-nixbot-gitlab> gitlab # [ 0.047547] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page316vm-test-run-nixbot-gitlab> gitlab # [ 0.059354] fbcon: Taking over console317vm-test-run-nixbot-gitlab> gitlab # [ 0.059364] ACPI: Interpreter disabled.318vm-test-run-nixbot-gitlab> gitlab # [ 0.060379] iommu: Default domain type: Translated319vm-test-run-nixbot-gitlab> gitlab # [ 0.060383] iommu: DMA domain TLB invalidation policy: strict mode320vm-test-run-nixbot-gitlab> gitlab # [ 0.069462] SCSI subsystem initialized321vm-test-run-nixbot-gitlab> gitlab # [ 0.069788] usbcore: registered new interface driver usbfs322vm-test-run-nixbot-gitlab> gitlab # [ 0.069811] usbcore: registered new interface driver hub323vm-test-run-nixbot-gitlab> gitlab # [ 0.069835] usbcore: registered new device driver usb324vm-test-run-nixbot-gitlab> gitlab # [ 0.070082] pps_core: LinuxPPS API ver. 1 registered325vm-test-run-nixbot-gitlab> gitlab # [ 0.070086] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>326vm-test-run-nixbot-gitlab> gitlab # [ 0.070094] PTP clock support registered327vm-test-run-nixbot-gitlab> gitlab # [ 0.070128] EDAC MC: Ver: 3.0.0328vm-test-run-nixbot-gitlab> gitlab # [ 0.070526] scmi_core: SCMI protocol bus registered329vm-test-run-nixbot-gitlab> gitlab # [ 0.071008] FPGA manager framework330vm-test-run-nixbot-gitlab> gitlab # [ 0.071864] vgaarb: loaded331vm-test-run-nixbot-gitlab> gitlab # [ 0.073832] clocksource: Switched to clocksource arch_sys_counter332vm-test-run-nixbot-gitlab> gitlab # [ 0.074313] VFS: Disk quotas dquot_6.6.0333vm-test-run-nixbot-gitlab> gitlab # [ 0.074336] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)334vm-test-run-nixbot-gitlab> gitlab # [ 0.074717] netfs: FS-Cache loaded335vm-test-run-nixbot-gitlab> gitlab # [ 0.074813] pnp: PnP ACPI: disabled336vm-test-run-nixbot-gitlab> gitlab # [ 0.077851] NET: Registered PF_INET protocol family337vm-test-run-nixbot-gitlab> gitlab # [ 0.078743] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)338vm-test-run-nixbot-gitlab> gitlab # [ 0.105413] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)339vm-test-run-nixbot-gitlab> gitlab # [ 0.105460] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)340vm-test-run-nixbot-gitlab> gitlab # [ 0.105501] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)341vm-test-run-nixbot-gitlab> gitlab # [ 0.105739] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [ 0.106330] TCP: Hash tables configured (established 65536 bind 65536)343vm-test-run-nixbot-gitlab> gitlab # [ 0.106418] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)344vm-test-run-nixbot-gitlab> gitlab # [ 0.106464] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)345vm-test-run-nixbot-gitlab> gitlab # [ 0.106575] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)346vm-test-run-nixbot-gitlab> gitlab # [ 0.106742] NET: Registered PF_UNIX/PF_LOCAL protocol family347vm-test-run-nixbot-gitlab> gitlab # [ 0.106758] NET: Registered PF_XDP protocol family348vm-test-run-nixbot-gitlab> gitlab # [ 0.106771] PCI: CLS 0 bytes, default 64349vm-test-run-nixbot-gitlab> gitlab # [ 0.107111] Trying to unpack rootfs image as initramfs...350vm-test-run-nixbot-gitlab> gitlab # [ 0.117901] kvm [1]: HYP mode not available351vm-test-run-nixbot-gitlab> gitlab # [ 0.159869] Initialise system trusted keyrings352vm-test-run-nixbot-gitlab> gitlab # [ 0.160027] workingset: timestamp_bits=42 max_order=21 bucket_order=0353vm-test-run-nixbot-gitlab> gitlab # [ 0.160543] squashfs: version 4.0 (2009/01/31) Phillip Lougher354vm-test-run-nixbot-gitlab> gitlab # [ 0.160605] 9p: Installing v9fs 9p2000 file system support355vm-test-run-nixbot-gitlab> gitlab # [ 0.172545] Key type asymmetric registered356vm-test-run-nixbot-gitlab> gitlab # [ 0.172550] Asymmetric key parser 'x509' registered357vm-test-run-nixbot-gitlab> gitlab # [ 0.172587] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)358vm-test-run-nixbot-gitlab> gitlab # [ 0.172751] io scheduler mq-deadline registered359vm-test-run-nixbot-gitlab> gitlab # [ 0.172755] io scheduler kyber registered360vm-test-run-nixbot-gitlab> gitlab # [ 0.181639] pl061_gpio 9030000.pl061: PL061 GPIO chip registered361vm-test-run-nixbot-gitlab> gitlab # [ 0.189872] ledtrig-cpu: registered to indicate activity on CPUs362vm-test-run-nixbot-gitlab> gitlab # [ 0.190262] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:363vm-test-run-nixbot-gitlab> gitlab # [ 0.190274] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000364vm-test-run-nixbot-gitlab> gitlab # [ 0.190284] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000365vm-test-run-nixbot-gitlab> gitlab # [ 0.190290] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000366vm-test-run-nixbot-gitlab> gitlab # [ 0.190307] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits367vm-test-run-nixbot-gitlab> gitlab # [ 0.190326] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]368vm-test-run-nixbot-gitlab> gitlab # [ 0.190389] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00369vm-test-run-nixbot-gitlab> gitlab # [ 0.190396] pci_bus 0000:00: root bus resource [bus 00-ff]370vm-test-run-nixbot-gitlab> gitlab # [ 0.190400] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]371vm-test-run-nixbot-gitlab> gitlab # [ 0.190403] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]372vm-test-run-nixbot-gitlab> gitlab # [ 0.190406] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]373vm-test-run-nixbot-gitlab> gitlab # [ 0.190503] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint374vm-test-run-nixbot-gitlab> gitlab # [ 0.191044] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint375vm-test-run-nixbot-gitlab> gitlab # [ 0.191272] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]376vm-test-run-nixbot-gitlab> gitlab # [ 0.191284] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]377vm-test-run-nixbot-gitlab> gitlab # [ 0.191311] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]378vm-test-run-nixbot-gitlab> gitlab # [ 0.191323] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]379vm-test-run-nixbot-gitlab> gitlab # [ 0.191877] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint380vm-test-run-nixbot-gitlab> gitlab # [ 0.192110] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]381vm-test-run-nixbot-gitlab> gitlab # [ 0.192122] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]382vm-test-run-nixbot-gitlab> gitlab # [ 0.192152] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]383vm-test-run-nixbot-gitlab> gitlab # [ 0.192710] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint384vm-test-run-nixbot-gitlab> gitlab # [ 0.192935] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]385vm-test-run-nixbot-gitlab> gitlab # [ 0.192947] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]386vm-test-run-nixbot-gitlab> gitlab # [ 0.192974] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]387vm-test-run-nixbot-gitlab> gitlab # [ 0.193539] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint388vm-test-run-nixbot-gitlab> gitlab # [ 0.193764] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]389vm-test-run-nixbot-gitlab> gitlab # [ 0.193776] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]390vm-test-run-nixbot-gitlab> gitlab # [ 0.193808] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]391vm-test-run-nixbot-gitlab> gitlab # [ 0.194370] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint392vm-test-run-nixbot-gitlab> gitlab # [ 0.194595] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]393vm-test-run-nixbot-gitlab> gitlab # [ 0.194607] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]394vm-test-run-nixbot-gitlab> gitlab # [ 0.194639] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]395vm-test-run-nixbot-gitlab> gitlab # [ 0.195188] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint396vm-test-run-nixbot-gitlab> gitlab # [ 0.195408] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]397vm-test-run-nixbot-gitlab> gitlab # [ 0.195420] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]398vm-test-run-nixbot-gitlab> gitlab # [ 0.195452] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]399vm-test-run-nixbot-gitlab> gitlab # [ 0.196017] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint400vm-test-run-nixbot-gitlab> gitlab # [ 0.196241] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]401vm-test-run-nixbot-gitlab> gitlab # [ 0.196253] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]402vm-test-run-nixbot-gitlab> gitlab # [ 0.196280] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]403vm-test-run-nixbot-gitlab> gitlab # [ 0.196291] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]404vm-test-run-nixbot-gitlab> gitlab # [ 0.196889] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint405vm-test-run-nixbot-gitlab> gitlab # [ 0.197114] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]406vm-test-run-nixbot-gitlab> gitlab # [ 0.197145] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]407vm-test-run-nixbot-gitlab> gitlab # [ 0.197705] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint408vm-test-run-nixbot-gitlab> gitlab # [ 0.197936] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]409vm-test-run-nixbot-gitlab> gitlab # [ 0.197968] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]410vm-test-run-nixbot-gitlab> gitlab # [ 0.198454] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint411vm-test-run-nixbot-gitlab> gitlab # [ 0.198682] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]412vm-test-run-nixbot-gitlab> gitlab # [ 0.198958] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint413vm-test-run-nixbot-gitlab> gitlab # [ 0.199238] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]414vm-test-run-nixbot-gitlab> gitlab # [ 0.199250] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]415vm-test-run-nixbot-gitlab> gitlab # [ 0.199277] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]416vm-test-run-nixbot-gitlab> gitlab # [ 0.199826] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint417vm-test-run-nixbot-gitlab> gitlab # [ 0.200051] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]418vm-test-run-nixbot-gitlab> gitlab # [ 0.200063] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]419vm-test-run-nixbot-gitlab> gitlab # [ 0.200093] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]420vm-test-run-nixbot-gitlab> gitlab # [ 0.200764] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned421vm-test-run-nixbot-gitlab> gitlab # [ 0.200771] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned422vm-test-run-nixbot-gitlab> gitlab # [ 0.200775] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned423vm-test-run-nixbot-gitlab> gitlab # [ 0.200823] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned424vm-test-run-nixbot-gitlab> gitlab # [ 0.200867] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned425vm-test-run-nixbot-gitlab> gitlab # [ 0.200916] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned426vm-test-run-nixbot-gitlab> gitlab # [ 0.200969] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned427vm-test-run-nixbot-gitlab> gitlab # [ 0.201020] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned428vm-test-run-nixbot-gitlab> gitlab # [ 0.201072] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned429vm-test-run-nixbot-gitlab> gitlab # [ 0.201122] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned430vm-test-run-nixbot-gitlab> gitlab # [ 0.201176] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned431vm-test-run-nixbot-gitlab> gitlab # [ 0.201226] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned432vm-test-run-nixbot-gitlab> gitlab # [ 0.201292] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned433vm-test-run-nixbot-gitlab> gitlab # [ 0.201340] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned434vm-test-run-nixbot-gitlab> gitlab # [ 0.201358] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned435vm-test-run-nixbot-gitlab> gitlab # [ 0.201376] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned436vm-test-run-nixbot-gitlab> gitlab # [ 0.201399] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned437vm-test-run-nixbot-gitlab> gitlab # [ 0.201422] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned438vm-test-run-nixbot-gitlab> gitlab # [ 0.201445] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned439vm-test-run-nixbot-gitlab> gitlab # [ 0.201469] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned440vm-test-run-nixbot-gitlab> gitlab # [ 0.201492] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned441vm-test-run-nixbot-gitlab> gitlab # [ 0.201511] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned442vm-test-run-nixbot-gitlab> gitlab # [ 0.201529] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned443vm-test-run-nixbot-gitlab> gitlab # [ 0.201547] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned444vm-test-run-nixbot-gitlab> gitlab # [ 0.201570] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned445vm-test-run-nixbot-gitlab> gitlab # [ 0.201588] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned446vm-test-run-nixbot-gitlab> gitlab # [ 0.201605] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned447vm-test-run-nixbot-gitlab> gitlab # [ 0.201628] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned448vm-test-run-nixbot-gitlab> gitlab # [ 0.201650] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned449vm-test-run-nixbot-gitlab> gitlab # [ 0.201672] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned450vm-test-run-nixbot-gitlab> gitlab # [ 0.201694] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned451vm-test-run-nixbot-gitlab> gitlab # [ 0.201717] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned452vm-test-run-nixbot-gitlab> gitlab # [ 0.201736] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned453vm-test-run-nixbot-gitlab> gitlab # [ 0.201753] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned454vm-test-run-nixbot-gitlab> gitlab # [ 0.201777] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]455vm-test-run-nixbot-gitlab> gitlab # [ 0.201782] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]456vm-test-run-nixbot-gitlab> gitlab # [ 0.201785] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]457vm-test-run-nixbot-gitlab> gitlab # [ 0.203286] pci 0000:00:0a.0: enabling device (0000 -> 0002)458vm-test-run-nixbot-gitlab> gitlab # [ 0.215123] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)459vm-test-run-nixbot-gitlab> gitlab # [ 0.217352] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)460vm-test-run-nixbot-gitlab> gitlab # [ 0.219529] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)461vm-test-run-nixbot-gitlab> gitlab # [ 0.221919] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)462vm-test-run-nixbot-gitlab> gitlab # [ 0.224270] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)463vm-test-run-nixbot-gitlab> gitlab # [ 0.227614] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)464vm-test-run-nixbot-gitlab> gitlab # [ 0.231809] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)465vm-test-run-nixbot-gitlab> gitlab # [ 0.233936] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)466vm-test-run-nixbot-gitlab> gitlab # [ 0.235948] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)467vm-test-run-nixbot-gitlab> gitlab # [ 0.238155] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)468vm-test-run-nixbot-gitlab> gitlab # [ 0.242109] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)469vm-test-run-nixbot-gitlab> gitlab # [ 0.249425] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled470vm-test-run-nixbot-gitlab> gitlab # [ 0.250968] msm_serial: driver initialized471vm-test-run-nixbot-gitlab> gitlab # [ 0.251095] SuperH (H)SCI(F) driver initialized472vm-test-run-nixbot-gitlab> gitlab # [ 0.251145] STM32 USART driver initialized473vm-test-run-nixbot-gitlab> gitlab # [ 0.272995] loop: module loaded474vm-test-run-nixbot-gitlab> gitlab # [ 0.273157] virtio_blk virtio4: 4/0/0 default/read/poll queues475vm-test-run-nixbot-gitlab> gitlab # [ 0.274941] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)476vm-test-run-nixbot-gitlab> gitlab # [ 0.276768] virtio_blk virtio5: 4/0/0 default/read/poll queues477vm-test-run-nixbot-gitlab> gitlab # [ 0.278532] virtio_blk virtio5: [vdb] 11826928 512-byte logical blocks (6.06 GB/5.64 GiB)478vm-test-run-nixbot-gitlab> gitlab # [ 0.280423] megasas: 07.734.00.00-rc1479vm-test-run-nixbot-gitlab> gitlab # [ 0.281024] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]480vm-test-run-nixbot-gitlab> gitlab # [ 0.285577] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000481vm-test-run-nixbot-gitlab> gitlab # [ 0.285628] Intel/Sharp Extended Query Table at 0x0031482vm-test-run-nixbot-gitlab> gitlab # [ 0.287250] Using buffer write method483vm-test-run-nixbot-gitlab> gitlab # [ 0.287277] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]484vm-test-run-nixbot-gitlab> gitlab # [ 0.289080] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000485vm-test-run-nixbot-gitlab> gitlab # [ 0.289093] Intel/Sharp Extended Query Table at 0x0031486vm-test-run-nixbot-gitlab> gitlab # [ 0.290653] Using buffer write method487vm-test-run-nixbot-gitlab> gitlab # [ 0.290666] Concatenating MTD devices:488vm-test-run-nixbot-gitlab> gitlab # [ 0.290668] (0): "0.flash"489vm-test-run-nixbot-gitlab> gitlab # [ 0.290671] (1): "0.flash"490vm-test-run-nixbot-gitlab> gitlab # [ 0.290673] into device "0.flash"491vm-test-run-nixbot-gitlab> gitlab # [ 0.547855] Freeing initrd memory: 25372K492vm-test-run-nixbot-gitlab> gitlab # [ 0.565563] tun: Universal TUN/TAP device driver, 1.6493vm-test-run-nixbot-gitlab> gitlab # [ 0.574227] thunder_xcv, ver 1.0494vm-test-run-nixbot-gitlab> gitlab # [ 0.574315] thunder_bgx, ver 1.0495vm-test-run-nixbot-gitlab> gitlab # [ 0.574429] nicpf, ver 1.0496vm-test-run-nixbot-gitlab> gitlab # [ 0.576022] e1000: Intel(R) PRO/1000 Network Driver497vm-test-run-nixbot-gitlab> gitlab # [ 0.576033] e1000: Copyright (c) 1999-2006 Intel Corporation.498vm-test-run-nixbot-gitlab> gitlab # [ 0.576213] e1000e: Intel(R) PRO/1000 Network Driver499vm-test-run-nixbot-gitlab> gitlab # [ 0.576230] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.500vm-test-run-nixbot-gitlab> gitlab # [ 0.576323] igb: Intel(R) Gigabit Ethernet Network Driver501vm-test-run-nixbot-gitlab> gitlab # [ 0.576333] igb: Copyright (c) 2007-2014 Intel Corporation.502vm-test-run-nixbot-gitlab> gitlab # [ 0.576399] igbvf: Intel(R) Gigabit Virtual Function Network Driver503vm-test-run-nixbot-gitlab> gitlab # [ 0.576410] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.504vm-test-run-nixbot-gitlab> gitlab # [ 0.576826] sky2: driver version 1.30505vm-test-run-nixbot-gitlab> gitlab # [ 0.581650] usbcore: registered new interface driver usb-storage506vm-test-run-nixbot-gitlab> gitlab # [ 0.581816] usbcore: registered new interface driver usbserial_generic507vm-test-run-nixbot-gitlab> gitlab # [ 0.581905] usbserial: USB Serial support registered for generic508vm-test-run-nixbot-gitlab> gitlab # [ 0.582069] ehci-pci 0000:00:0a.0: EHCI Host Controller509vm-test-run-nixbot-gitlab> gitlab # [ 0.582140] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1510vm-test-run-nixbot-gitlab> gitlab # [ 0.582825] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000511vm-test-run-nixbot-gitlab> gitlab # [ 0.583665] hv_vmbus: registering driver hyperv_keyboard512vm-test-run-nixbot-gitlab> gitlab # [ 0.586368] rtc-pl031 9010000.pl031: registered as rtc0513vm-test-run-nixbot-gitlab> gitlab # [ 0.586428] rtc-pl031 9010000.pl031: setting system clock to 2026-06-29T21:54:28 UTC (1782770068)514vm-test-run-nixbot-gitlab> gitlab # [ 0.587345] i2c_dev: i2c /dev entries driver515vm-test-run-nixbot-gitlab> gitlab # [ 0.593943] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00516vm-test-run-nixbot-gitlab> gitlab # [ 0.595331] hub 1-0:1.0: USB hub found517vm-test-run-nixbot-gitlab> gitlab # [ 0.595394] hub 1-0:1.0: 6 ports detected518vm-test-run-nixbot-gitlab> gitlab # [ 0.595643] sdhci: Secure Digital Host Controller Interface driver519vm-test-run-nixbot-gitlab> gitlab # [ 0.595656] sdhci: Copyright(c) Pierre Ossman520vm-test-run-nixbot-gitlab> gitlab # [ 0.596469] Synopsys Designware Multimedia Card Interface Driver521vm-test-run-nixbot-gitlab> gitlab # [ 0.597641] sdhci-pltfm: SDHCI platform and OF driver helper522vm-test-run-nixbot-gitlab> gitlab # [ 0.602508] hid: raw HID events driver (C) Jiri Kosina523vm-test-run-nixbot-gitlab> gitlab # [ 0.603447] usbcore: registered new interface driver usbhid524vm-test-run-nixbot-gitlab> gitlab # [ 0.603459] usbhid: USB HID core driver525vm-test-run-nixbot-gitlab> gitlab # [ 0.648900] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available526vm-test-run-nixbot-gitlab> gitlab # [ 0.653324] drop_monitor: Initializing network drop monitor service527vm-test-run-nixbot-gitlab> gitlab # [ 0.653685] NET: Registered PF_INET6 protocol family528vm-test-run-nixbot-gitlab> gitlab # [ 0.655423] Segment Routing with IPv6529vm-test-run-nixbot-gitlab> gitlab # [ 0.655455] In-situ OAM (IOAM) with IPv6530vm-test-run-nixbot-gitlab> gitlab # [ 0.655537] NET: Registered PF_PACKET protocol family531vm-test-run-nixbot-gitlab> gitlab # [ 0.655827] 9pnet: Installing 9P2000 support532vm-test-run-nixbot-gitlab> gitlab # [ 0.659891] Key type dns_resolver registered533vm-test-run-nixbot-gitlab> gitlab # [ 0.678727] registered taskstats version 1534vm-test-run-nixbot-gitlab> gitlab # [ 0.679262] Loading compiled-in X.509 certificates535vm-test-run-nixbot-gitlab> gitlab # [ 0.702471] Demotion targets for Node 0: null536vm-test-run-nixbot-gitlab> gitlab # [ 0.702726] Key type .fscrypt registered537vm-test-run-nixbot-gitlab> gitlab # [ 0.702733] Key type fscrypt-provisioning registered538vm-test-run-nixbot-gitlab> gitlab # [ 0.702926] ima: No TPM chip found, activating TPM-bypass!539vm-test-run-nixbot-gitlab> gitlab # [ 0.702950] ima: Allocated hash algorithm: sha1540vm-test-run-nixbot-gitlab> gitlab # [ 0.702976] ima: No architecture policies found541vm-test-run-nixbot-gitlab> gitlab # [ 0.704223] input: gpio-keys as /devices/platform/gpio-keys/input/input0542vm-test-run-nixbot-gitlab> gitlab # [ 0.731615] clk: Disabling unused clocks543vm-test-run-nixbot-gitlab> gitlab # [ 0.731646] PM: genpd: Disabling unused power domains544vm-test-run-nixbot-gitlab> gitlab # [ 0.737668] Freeing unused kernel memory: 4736K545vm-test-run-nixbot-gitlab> gitlab # [ 0.737936] Run /init as init process546vm-test-run-nixbot-gitlab> gitlab # [ 0.773978] systemd[1]: Successfully made /usr/ read-only.547vm-test-run-nixbot-gitlab> gitlab # [ 0.853962] usb 1-1: new high-speed USB device number 2 using ehci-pci548vm-test-run-nixbot-gitlab> gitlab # [ 1.012767] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1549vm-test-run-nixbot-gitlab> gitlab # [ 1.098530] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0550vm-test-run-nixbot-gitlab> gitlab # [ 1.108632] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)551vm-test-run-nixbot-gitlab> gitlab # [ 1.108697] systemd[1]: Detected virtualization qemu.552vm-test-run-nixbot-gitlab> gitlab # [ 1.108767] systemd[1]: Detected architecture arm64.553vm-test-run-nixbot-gitlab> gitlab # [ 1.108783] systemd[1]: Running in initrd.554vm-test-run-nixbot-gitlab> gitlab # [ 1.109738] systemd[1]: Initializing machine ID from random generator.555vm-test-run-nixbot-gitlab> gitlab # [ 1.110159] systemd[1]: Hostname set to <gitlab>.556vm-test-run-nixbot-gitlab> gitlab # [ 1.221939] usb 1-2: new high-speed USB device number 3 using ehci-pci557vm-test-run-nixbot-gitlab> gitlab # [ 1.278856] systemd[1]: Queued start job for default target Initrd Default Target.558vm-test-run-nixbot-gitlab> gitlab # [ 1.380102] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2559vm-test-run-nixbot-gitlab> gitlab # [ 1.380689] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0560vm-test-run-nixbot-gitlab> gitlab # [ 1.520556] systemd[1]: Created slice Slice /system/modprobe.561vm-test-run-nixbot-gitlab> gitlab # [ 1.521100] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.562vm-test-run-nixbot-gitlab> gitlab # [ 1.521197] systemd[1]: Expecting device /dev/disk/by-label/nix-store...563vm-test-run-nixbot-gitlab> gitlab # [ 1.521230] systemd[1]: Expecting device /dev/disk/by-label/nixos...564vm-test-run-nixbot-gitlab> gitlab # [ 1.521288] systemd[1]: Reached target Path Units.565vm-test-run-nixbot-gitlab> gitlab # [ 1.521329] systemd[1]: Reached target Slice Units.566vm-test-run-nixbot-gitlab> gitlab # [ 1.521369] systemd[1]: Reached target Swaps.567vm-test-run-nixbot-gitlab> gitlab # [ 1.521409] systemd[1]: Reached target Timer Units.568vm-test-run-nixbot-gitlab> gitlab # [ 1.521774] systemd[1]: Listening on D-Bus System Message Bus Socket.569vm-test-run-nixbot-gitlab> gitlab # [ 1.522206] systemd[1]: Listening on Journal Socket (/dev/log).570vm-test-run-nixbot-gitlab> gitlab # [ 1.522525] systemd[1]: Listening on Journal Sockets.571vm-test-run-nixbot-gitlab> gitlab # [ 1.522850] systemd[1]: Listening on udev Control Socket.572vm-test-run-nixbot-gitlab> gitlab # [ 1.523148] systemd[1]: Listening on udev Kernel Socket.573vm-test-run-nixbot-gitlab> gitlab # [ 1.523192] systemd[1]: Reached target Socket Units.574vm-test-run-nixbot-gitlab> gitlab # [ 1.530663] systemd[1]: Starting Create List of Static Device Nodes...575vm-test-run-nixbot-gitlab> gitlab # [ 1.538847] systemd[1]: Starting Load Kernel Module 9pnet_virtio...576vm-test-run-nixbot-gitlab> gitlab # [ 1.538995] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs577vm-test-run-nixbot-gitlab> gitlab # [ 1.570762] systemd[1]: Mounting Kernel Configuration File System...578vm-test-run-nixbot-gitlab> gitlab # [ 1.584293] systemd[1]: Starting Journal Service...579vm-test-run-nixbot-gitlab> gitlab # [ 1.592431] systemd[1]: Starting Load Kernel Modules...580vm-test-run-nixbot-gitlab> gitlab # [ 1.592586] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki581vm-test-run-nixbot-gitlab> gitlab # [ 1.605600] systemd[1]: Starting Coldplug All udev Devices...582vm-test-run-nixbot-gitlab> gitlab # [ 1.630507] systemd[1]: Finished Create List of Static Device Nodes.583vm-test-run-nixbot-gitlab> gitlab # [ 1.633074] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.584vm-test-run-nixbot-gitlab> gitlab # [ 1.638284] systemd-journald[154]: Collecting audit messages is disabled.585vm-test-run-nixbot-gitlab> gitlab # [ 1.639883] systemd[1]: Finished Load Kernel Module 9pnet_virtio.586vm-test-run-nixbot-gitlab> gitlab # [ 1.650770] systemd[1]: Mounted Kernel Configuration File System.587vm-test-run-nixbot-gitlab> gitlab # [ 1.651994] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0588vm-test-run-nixbot-gitlab> gitlab # [ 1.652389] [drm] features: -virgl +edid -resource_blob -host_visible589vm-test-run-nixbot-gitlab> gitlab # [ 1.652394] [drm] features: -context_init590vm-test-run-nixbot-gitlab> gitlab # [ 1.653683] [drm] number of scanouts: 1591vm-test-run-nixbot-gitlab> gitlab # [ 1.653710] [drm] number of cap sets: 0592vm-test-run-nixbot-gitlab> gitlab # [ 1.665882] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic593vm-test-run-nixbot-gitlab> gitlab # [ 1.665916] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0594vm-test-run-nixbot-gitlab> gitlab # [ 1.666858] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...595vm-test-run-nixbot-gitlab> gitlab # [ 1.667120] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.596vm-test-run-nixbot-gitlab> gitlab # [ 1.670150] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev597vm-test-run-nixbot-gitlab> gitlab # [ 1.681090] Console: switching to colour frame buffer device 160x50598vm-test-run-nixbot-gitlab> gitlab # [ 1.688115] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device599vm-test-run-nixbot-gitlab> gitlab # [ 1.706383] systemd[1]: Finished Load Kernel Modules.600vm-test-run-nixbot-gitlab> gitlab # [ 1.717982] systemd[1]: Starting Apply Kernel Variables...601vm-test-run-nixbot-gitlab> gitlab # [ 1.718464] systemd[1]: Started Journal Service.602vm-test-run-nixbot-gitlab> gitlab # [ 1.716724] systemd-modules-load[157]: Using 4 probe threads603vm-test-run-nixbot-gitlab> gitlab # [ 1.722325] systemd-modules-load[157]: Module 'virtio_balloon' is built in604vm-test-run-nixbot-gitlab> gitlab # [ 1.723440] systemd-modules-load[157]: Module 'virtio_console' is built in605vm-test-run-nixbot-gitlab> gitlab # [ 1.724550] systemd-modules-load[157]: Module 'virtio_rng' is built in606vm-test-run-nixbot-gitlab> gitlab # [ 1.725577] systemd-modules-load[157]: Inserted module 'dm_mod'607vm-test-run-nixbot-gitlab> gitlab # [ 1.726534] systemd-modules-load[157]: Inserted module 'virtio_gpu'608vm-test-run-nixbot-gitlab> gitlab # [ 1.727537] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.609vm-test-run-nixbot-gitlab> gitlab # [ 1.738608] systemd[1]: Starting Create Static Device Nodes in /dev...610vm-test-run-nixbot-gitlab> gitlab # [ 1.748207] systemd[1]: Finished Apply Kernel Variables.611vm-test-run-nixbot-gitlab> gitlab # [ 1.757479] systemd[1]: Finished Create Static Device Nodes in /dev.612vm-test-run-nixbot-gitlab> gitlab # [ 1.759131] systemd[1]: Reached target Preparation for Local File Systems.613vm-test-run-nixbot-gitlab> gitlab # [ 1.761368] systemd[1]: Reached target Local File Systems.614vm-test-run-nixbot-gitlab> gitlab # [ 1.766114] systemd[1]: Starting Create System Files and Directories...615vm-test-run-nixbot-gitlab> gitlab # [ 1.775471] systemd[1]: Starting Rule-based Manager for Device Events and Files...616vm-test-run-nixbot-gitlab> gitlab # [ 1.791600] systemd[1]: Finished Create System Files and Directories.617vm-test-run-nixbot-gitlab> gitlab # [ 1.810918] systemd-udevd[194]: Using default interface naming scheme 'v260'.618vm-test-run-nixbot-gitlab> gitlab # [ 1.829492] systemd[1]: Started Rule-based Manager for Device Events and Files.619vm-test-run-nixbot-gitlab> gitlab # [ 1.871863] systemd[1]: Starting Virtual Console Setup...620vm-test-run-nixbot-gitlab> gitlab # [ 1.946353] systemd-vconsole-setup[210]: Configuration of first virtual console was skipped, ignoring remaining ones.621vm-test-run-nixbot-gitlab> gitlab # [ 1.952449] systemd[1]: Finished Virtual Console Setup.622vm-test-run-nixbot-gitlab> gitlab # [ 2.392649] systemd[1]: Finished Coldplug All udev Devices.623vm-test-run-nixbot-gitlab> gitlab # [ 2.393529] systemd[1]: Reached target System Initialization.624vm-test-run-nixbot-gitlab> gitlab # [ 2.394372] systemd[1]: Reached target Basic System.625vm-test-run-nixbot-gitlab> gitlab # [ 2.568303] (udev-worker)[220]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.626vm-test-run-nixbot-gitlab> gitlab # [ 2.572224] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line.627vm-test-run-nixbot-gitlab> gitlab # [ 2.573793] (udev-worker)[223]: Network interface NamePolicy= disabled on kernel command line.628vm-test-run-nixbot-gitlab> gitlab # [ 2.603522] systemd[1]: Found device /dev/disk/by-label/nix-store.629vm-test-run-nixbot-gitlab> gitlab # [ 2.612156] systemd[1]: Found device /dev/disk/by-label/nixos.630vm-test-run-nixbot-gitlab> gitlab # [ 2.624422] systemd[1]: Reached target Initrd Root Device.631vm-test-run-nixbot-gitlab> gitlab # [ 2.629228] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...632vm-test-run-nixbot-gitlab> gitlab # [ 2.670223] systemd-fsck[235]: nixos: clean, 12/65536 files, 13019/262144 blocks633vm-test-run-nixbot-gitlab> gitlab # [ 2.676412] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.634vm-test-run-nixbot-gitlab> gitlab # [ 2.680302] systemd[1]: Mounting /sysroot...635vm-test-run-nixbot-gitlab> gitlab # [ 2.715954] EXT4-fs (vda): mounted filesystem 91c97e3c-8f24-4d1f-ae7a-ef10c3ffb8f2 r/w with ordered data mode. Quota mode: none.636vm-test-run-nixbot-gitlab> gitlab # [ 2.717125] systemd[1]: Mounted /sysroot.637vm-test-run-nixbot-gitlab> gitlab # [ 2.718669] systemd[1]: Reached target Initrd Root File System.638vm-test-run-nixbot-gitlab> gitlab # [ 2.723387] systemd[1]: Mounting /sysroot/nix/.ro-store...639vm-test-run-nixbot-gitlab> gitlab # [ 2.728399] systemd[1]: Mounting /sysroot/nix/.rw-store...640vm-test-run-nixbot-gitlab> gitlab # [ 2.741910] systemd[1]: Mounting /sysroot/run...641vm-test-run-nixbot-gitlab> gitlab # [ 2.749073] systemd[1]: Mounting /sysroot/tmp/shared...642vm-test-run-nixbot-gitlab> gitlab # [ 2.766282] systemd[1]: Mounting /sysroot/tmp/xchg...643vm-test-run-nixbot-gitlab> gitlab # [ 2.798922] systemd[1]: Starting Mountpoints Configured in the Real Root...644vm-test-run-nixbot-gitlab> gitlab # [ 2.812254] systemd-sysroot-fstab-check[258]: /sysroot should be mounted in the initrd, will request daemon-reload.645vm-test-run-nixbot-gitlab> gitlab # [ 2.822016] systemd[1]: Mounted /sysroot/nix/.rw-store.646vm-test-run-nixbot-gitlab> gitlab # [ 2.826942] systemd[1]: Mounted /sysroot/run.647vm-test-run-nixbot-gitlab> gitlab # [ 2.851571] systemd[1]: Starting rw-sysroot-nix-store.service...648vm-test-run-nixbot-gitlab> gitlab # [ 2.855437] erofs (device vdb): mounted with root inode @ nid 128.649vm-test-run-nixbot-gitlab> gitlab # [ 2.861477] systemd[1]: Mounted /sysroot/tmp/shared.650vm-test-run-nixbot-gitlab> gitlab # [ 2.878459] systemd[1]: Mounted /sysroot/nix/.ro-store.651vm-test-run-nixbot-gitlab> gitlab # [ 2.882446] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.652vm-test-run-nixbot-gitlab> gitlab # [ 2.887848] systemd[1]: Finished rw-sysroot-nix-store.service.653vm-test-run-nixbot-gitlab> gitlab # [ 2.894527] systemd[1]: Reload requested from client PID 258 ('systemd-sysroot') (unit initrd-parse-etc.service)...654vm-test-run-nixbot-gitlab> gitlab # [ 2.896619] systemd[1]: Reloading...655vm-test-run-nixbot-gitlab> gitlab # [ 3.188985] systemd[1]: Reloading finished in 293 ms.656vm-test-run-nixbot-gitlab> gitlab # [ 3.223175] (udev-worker)[227]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory657vm-test-run-nixbot-gitlab> gitlab # [ 3.227532] (udev-worker)[227]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory658vm-test-run-nixbot-gitlab> gitlab # [ 3.243149] systemd-sysroot-fstab-check[258]: Requesting initrd-fs.target/start/replace...659vm-test-run-nixbot-gitlab> gitlab # [ 3.336870] systemd[1]: Mounted /sysroot/tmp/xchg.660vm-test-run-nixbot-gitlab> gitlab # [ 3.346445] systemd-sysroot-fstab-check[258]: Requesting swap.target/start/replace...661vm-test-run-nixbot-gitlab> gitlab # [ 3.352424] systemd[1]: initrd-parse-etc.service: Deactivated successfully.662vm-test-run-nixbot-gitlab> gitlab # [ 3.356990] systemd[1]: Finished Mountpoints Configured in the Real Root.663vm-test-run-nixbot-gitlab> gitlab # [ 3.359982] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.664vm-test-run-nixbot-gitlab> gitlab # [ 3.382585] systemd[1]: Starting rw-sysroot-nix-store.service...665vm-test-run-nixbot-gitlab> gitlab # [ 3.385425] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.666vm-test-run-nixbot-gitlab> gitlab # [ 3.388697] systemd[1]: Stopped Virtual Console Setup.667vm-test-run-nixbot-gitlab> gitlab # [ 3.391026] systemd[1]: Stopping Virtual Console Setup...668vm-test-run-nixbot-gitlab> gitlab # [ 3.393495] systemd[1]: Starting Virtual Console Setup...669vm-test-run-nixbot-gitlab> gitlab # [ 3.444683] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.670vm-test-run-nixbot-gitlab> gitlab # [ 3.450402] systemd[1]: Finished rw-sysroot-nix-store.service.671vm-test-run-nixbot-gitlab> gitlab # [ 3.475541] systemd-vconsole-setup[488]: Configuration of first virtual console was skipped, ignoring remaining ones.672vm-test-run-nixbot-gitlab> gitlab # [ 3.482788] systemd[1]: Finished Virtual Console Setup.673vm-test-run-nixbot-gitlab> gitlab # [ 3.628551] systemd[1]: Mounting /sysroot/nix/store...674vm-test-run-nixbot-gitlab> gitlab # [ 3.693892] systemd[1]: Mounted /sysroot/nix/store.675vm-test-run-nixbot-gitlab> gitlab # [ 3.696282] systemd[1]: Reached target Initrd File Systems.676vm-test-run-nixbot-gitlab> gitlab # [ 3.702776] systemd[1]: Starting Find NixOS closure...677vm-test-run-nixbot-gitlab> gitlab # [ 3.710161] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...678vm-test-run-nixbot-gitlab> gitlab # [ 3.759934] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.679vm-test-run-nixbot-gitlab> gitlab # [ 3.763587] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.680vm-test-run-nixbot-gitlab> gitlab # [ 3.786574] systemd[1]: Finished Find NixOS closure.681vm-test-run-nixbot-gitlab> gitlab # [ 3.788870] systemd[1]: Reached target Initrd Default Target.682vm-test-run-nixbot-gitlab> gitlab # [ 3.795693] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...683vm-test-run-nixbot-gitlab> gitlab # [ 3.844681] systemd[1]: Stopped target Initrd Default Target.684vm-test-run-nixbot-gitlab> gitlab # [ 3.847231] systemd[1]: Stopped target Basic System.685vm-test-run-nixbot-gitlab> gitlab # [ 3.849638] systemd[1]: Stopped target Initrd Root Device.686vm-test-run-nixbot-gitlab> gitlab # [ 3.852110] systemd[1]: Stopped target Path Units.687vm-test-run-nixbot-gitlab> gitlab # [ 3.854301] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.688vm-test-run-nixbot-gitlab> gitlab # [ 3.857657] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.689vm-test-run-nixbot-gitlab> gitlab # [ 3.861080] systemd[1]: Stopped target Slice Units.690vm-test-run-nixbot-gitlab> gitlab # [ 3.863288] systemd[1]: Stopped target Socket Units.691vm-test-run-nixbot-gitlab> gitlab # [ 3.865640] systemd[1]: Stopped target System Initialization.692vm-test-run-nixbot-gitlab> gitlab # [ 3.868237] systemd[1]: Stopped target Swaps.693vm-test-run-nixbot-gitlab> gitlab # [ 3.870255] systemd[1]: Stopped target Timer Units.694vm-test-run-nixbot-gitlab> gitlab # [ 3.872480] systemd[1]: dbus.socket: Deactivated successfully.695vm-test-run-nixbot-gitlab> gitlab # [ 3.875016] systemd[1]: Closed D-Bus System Message Bus Socket.696vm-test-run-nixbot-gitlab> gitlab # [ 3.877743] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.697vm-test-run-nixbot-gitlab> gitlab # [ 3.881054] systemd[1]: Stopped Find NixOS closure.698vm-test-run-nixbot-gitlab> gitlab # [ 3.883345] systemd[1]: Starting Load Kernel Module 9pnet_virtio...699vm-test-run-nixbot-gitlab> gitlab # [ 3.885942] systemd[1]: Starting rw-sysroot-nix-store.service...700vm-test-run-nixbot-gitlab> gitlab # [ 3.888406] systemd[1]: systemd-sysctl.service: Deactivated successfully.701vm-test-run-nixbot-gitlab> gitlab # [ 3.891124] systemd[1]: Stopped Apply Kernel Variables.702vm-test-run-nixbot-gitlab> gitlab # [ 3.893353] systemd[1]: systemd-modules-load.service: Deactivated successfully.703vm-test-run-nixbot-gitlab> gitlab # [ 3.896214] systemd[1]: Stopped Load Kernel Modules.704vm-test-run-nixbot-gitlab> gitlab # [ 3.898266] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.705vm-test-run-nixbot-gitlab> gitlab # [ 3.901493] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.706vm-test-run-nixbot-gitlab> gitlab # [ 3.904610] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.707vm-test-run-nixbot-gitlab> gitlab # [ 3.907510] systemd[1]: Stopped Create System Files and Directories.708vm-test-run-nixbot-gitlab> gitlab # [ 3.910712] systemd[1]: Stopped target Local File Systems.709vm-test-run-nixbot-gitlab> gitlab # [ 3.912640] systemd[1]: Stopped target Preparation for Local File Systems.710vm-test-run-nixbot-gitlab> gitlab # [ 3.914910] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.711vm-test-run-nixbot-gitlab> gitlab # [ 3.917493] systemd[1]: Stopped Coldplug All udev Devices.712vm-test-run-nixbot-gitlab> gitlab # [ 3.919350] systemd[1]: Stopping Rule-based Manager for Device Events and Files...713vm-test-run-nixbot-gitlab> gitlab # [ 3.921872] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [ 3.924297] systemd[1]: Stopped Virtual Console Setup.715vm-test-run-nixbot-gitlab> gitlab # [ 3.933069] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [ 3.936438] systemd[1]: Finished Load Kernel Module 9pnet_virtio.717vm-test-run-nixbot-gitlab> gitlab # [ 3.941411] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.718vm-test-run-nixbot-gitlab> gitlab # [ 3.945116] systemd[1]: Finished rw-sysroot-nix-store.service.719vm-test-run-nixbot-gitlab> gitlab # [ 3.959425] systemd[1]: systemd-udevd.service: Deactivated successfully.720vm-test-run-nixbot-gitlab> gitlab # [ 3.964565] systemd[1]: Stopped Rule-based Manager for Device Events and Files.721vm-test-run-nixbot-gitlab> gitlab # [ 3.967096] systemd[1]: systemd-udevd.service: Consumed 2.320s CPU time over 2.184s wall clock time, 32.3M memory peak.722vm-test-run-nixbot-gitlab> gitlab # [ 3.974375] systemd[1]: initrd-cleanup.service: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [ 3.978996] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.724vm-test-run-nixbot-gitlab> gitlab # [ 3.996146] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.725vm-test-run-nixbot-gitlab> gitlab # [ 3.999255] systemd[1]: Closed udev Control Socket.726vm-test-run-nixbot-gitlab> gitlab # [ 4.011678] systemd[1]: Starting Cleanup udev Database...727vm-test-run-nixbot-gitlab> gitlab # [ 4.014586] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.728vm-test-run-nixbot-gitlab> gitlab # [ 4.018090] systemd[1]: Stopped Create Static Device Nodes in /dev.729vm-test-run-nixbot-gitlab> gitlab # [ 4.020935] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [ 4.024475] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.731vm-test-run-nixbot-gitlab> gitlab # [ 4.028921] systemd[1]: kmod-static-nodes.service: Deactivated successfully.732vm-test-run-nixbot-gitlab> gitlab # [ 4.031704] systemd[1]: Stopped Create List of Static Device Nodes.733vm-test-run-nixbot-gitlab> gitlab # [ 4.110959] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.734vm-test-run-nixbot-gitlab> gitlab # [ 4.114405] systemd[1]: Finished Cleanup udev Database.735vm-test-run-nixbot-gitlab> gitlab # [ 4.123464] systemd[1]: Reached target Switch Root.736vm-test-run-nixbot-gitlab> gitlab # [ 4.132585] systemd[1]: Starting NixOS Activation...737vm-test-run-nixbot-gitlab> gitlab # [ 4.322376] initrd-nixos-activation-start[573]: booting system configuration /nix/store/99x979qwsg5j0ap0mklhjcwk21as8gkv-nixos-system-gitlab-test738vm-test-run-nixbot-gitlab> gitlab # [ 4.392757] initrd-nixos-activation-start[573]: running activation script...739vm-test-run-nixbot-gitlab> gitlab # [ 4.726826] initrd-nixos-activation-start[596]: setting up /etc...740vm-test-run-nixbot-gitlab> gitlab # [ 4.893917] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.741vm-test-run-nixbot-gitlab> gitlab # [ 4.898430] systemd[1]: Finished NixOS Activation.742vm-test-run-nixbot-gitlab> gitlab # [ 4.915603] systemd[1]: Starting Switch Root...743vm-test-run-nixbot-gitlab> gitlab # [ 4.970762] systemd[1]: Switching root.744vm-test-run-nixbot-gitlab> gitlab # [ 5.624637] systemd-journald[154]: Received SIGTERM from PID 1 (systemd).745vm-test-run-nixbot-gitlab> gitlab # [ 6.304635] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)746vm-test-run-nixbot-gitlab> gitlab # [ 6.304778] systemd[1]: Detected virtualization qemu.747vm-test-run-nixbot-gitlab> gitlab # [ 6.304863] systemd[1]: Detected architecture arm64.748vm-test-run-nixbot-gitlab> gitlab # [ 6.305024] systemd[1]: Detected first boot.749vm-test-run-nixbot-gitlab> gitlab # [ 6.306623] systemd[1]: Initializing machine ID from random generator.750vm-test-run-nixbot-gitlab> gitlab # [ 6.558833] systemd[1]: bpf-restrict-fs: LSM BPF program attached751vm-test-run-nixbot-gitlab> gitlab # [ 6.670592] systemd[1]: Applying preset policy.752vm-test-run-nixbot-gitlab> gitlab # [ 6.897151] systemd[1]: Populated /etc with preset unit settings.753vm-test-run-nixbot-gitlab> gitlab # [ 7.052979] systemd[1]: initrd-switch-root.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [ 7.053238] systemd[1]: Stopped initrd-switch-root.service.755vm-test-run-nixbot-gitlab> gitlab # [ 7.054914] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.756vm-test-run-nixbot-gitlab> gitlab # [ 7.062543] systemd[1]: Created slice Slice /system/getty.757vm-test-run-nixbot-gitlab> gitlab # [ 7.066123] systemd[1]: Created slice GitLab DevOps Platform Slice.758vm-test-run-nixbot-gitlab> gitlab # [ 7.070885] systemd[1]: Created slice User and Session Slice.759vm-test-run-nixbot-gitlab> gitlab # [ 7.071530] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.760vm-test-run-nixbot-gitlab> gitlab # [ 7.072351] systemd[1]: Started Forward Password Requests to Wall Directory Watch.761vm-test-run-nixbot-gitlab> gitlab # [ 7.073587] systemd[1]: Expecting device /dev/hvc0...762vm-test-run-nixbot-gitlab> gitlab # [ 7.074739] systemd[1]: Expecting device /dev/ttyAMA0...763vm-test-run-nixbot-gitlab> gitlab # [ 7.075872] systemd[1]: Reached target Local Encrypted Volumes.764vm-test-run-nixbot-gitlab> gitlab # [ 7.077035] systemd[1]: Stopped target initrd-fs.target.765vm-test-run-nixbot-gitlab> gitlab # [ 7.078159] systemd[1]: Stopped target initrd-root-fs.target.766vm-test-run-nixbot-gitlab> gitlab # [ 7.079219] systemd[1]: Stopped target initrd-switch-root.target.767vm-test-run-nixbot-gitlab> gitlab # [ 7.080320] systemd[1]: Reached target Virtual Machines and Containers.768vm-test-run-nixbot-gitlab> gitlab # [ 7.081455] systemd[1]: Reached target Path Units.769vm-test-run-nixbot-gitlab> gitlab # [ 7.082595] systemd[1]: Reached target Remote File Systems.770vm-test-run-nixbot-gitlab> gitlab # [ 7.083686] systemd[1]: Reached target Slice Units.771vm-test-run-nixbot-gitlab> gitlab # [ 7.084832] systemd[1]: Reached target Swaps.772vm-test-run-nixbot-gitlab> gitlab # [ 7.089328] systemd[1]: Listening on Process Core Dump Socket.773vm-test-run-nixbot-gitlab> gitlab # [ 7.092829] systemd[1]: Listening on Credential Encryption/Decryption.774vm-test-run-nixbot-gitlab> gitlab # [ 7.103576] systemd[1]: Starting Journal Log Access Socket...775vm-test-run-nixbot-gitlab> gitlab # [ 7.106286] systemd[1]: Listening on Journal Audit Socket.776vm-test-run-nixbot-gitlab> gitlab # [ 7.108167] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.777vm-test-run-nixbot-gitlab> gitlab # [ 7.108992] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki778vm-test-run-nixbot-gitlab> gitlab # [ 7.110399] systemd[1]: Listening on udev Control Socket.779vm-test-run-nixbot-gitlab> gitlab # [ 7.151148] systemd[1]: Mounting Huge Pages File System...780vm-test-run-nixbot-gitlab> gitlab # [ 7.158447] systemd[1]: Mounting POSIX Message Queue File System...781vm-test-run-nixbot-gitlab> gitlab # [ 7.165142] systemd[1]: Mounting Kernel Debug File System...782vm-test-run-nixbot-gitlab> gitlab # [ 7.175600] systemd[1]: Mounting Kernel Trace File System...783vm-test-run-nixbot-gitlab> gitlab # [ 7.187066] systemd[1]: Starting Create List of Static Device Nodes...784vm-test-run-nixbot-gitlab> gitlab # [ 7.196051] systemd[1]: Starting Load Kernel Module 9pnet_virtio...785vm-test-run-nixbot-gitlab> gitlab # [ 7.198216] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs786vm-test-run-nixbot-gitlab> gitlab # [ 7.211961] systemd[1]: Mounting Kernel Configuration File System...787vm-test-run-nixbot-gitlab> gitlab # [ 7.214270] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm788vm-test-run-nixbot-gitlab> gitlab # [ 7.218131] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore789vm-test-run-nixbot-gitlab> gitlab # [ 7.238138] systemd[1]: Starting Load Kernel Module fuse...790vm-test-run-nixbot-gitlab> gitlab # [ 7.239680] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67791vm-test-run-nixbot-gitlab> gitlab # [ 7.249195] systemd[1]: Starting Journal Service...792vm-test-run-nixbot-gitlab> gitlab # [ 7.255043] systemd[1]: Starting Load Kernel Modules...793vm-test-run-nixbot-gitlab> gitlab # [ 7.272573] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...794vm-test-run-nixbot-gitlab> gitlab # [ 7.287089] systemd[1]: Starting Remount Root and Kernel File Systems...795vm-test-run-nixbot-gitlab> gitlab # [ 7.290790] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki796vm-test-run-nixbot-gitlab> gitlab # [ 7.306021] systemd[1]: Starting Coldplug All udev Devices...797vm-test-run-nixbot-gitlab> gitlab # [ 7.316977] systemd[1]: Listening on Journal Log Access Socket.798vm-test-run-nixbot-gitlab> gitlab # [ 7.322261] systemd[1]: Mounted Huge Pages File System.799vm-test-run-nixbot-gitlab> gitlab # [ 7.323220] fuse: init (API version 7.45)800vm-test-run-nixbot-gitlab> gitlab # [ 7.324159] systemd[1]: Mounted POSIX Message Queue File System.801vm-test-run-nixbot-gitlab> gitlab # [ 7.325569] systemd[1]: Mounted Kernel Debug File System.802vm-test-run-nixbot-gitlab> gitlab # [ 7.327007] systemd-journald[805]: Collecting audit messages is enabled.803vm-test-run-nixbot-gitlab> gitlab # [ 7.327048] systemd[1]: Mounted Kernel Trace File System.804vm-test-run-nixbot-gitlab> gitlab # [ 7.328919] systemd[1]: Finished Create List of Static Device Nodes.805vm-test-run-nixbot-gitlab> gitlab # [ 7.330457] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.806vm-test-run-nixbot-gitlab> gitlab # [ 7.332090] systemd[1]: Finished Load Kernel Module 9pnet_virtio.807vm-test-run-nixbot-gitlab> gitlab # [ 7.333479] systemd[1]: Mounted Kernel Configuration File System.808vm-test-run-nixbot-gitlab> gitlab # [ 7.334913] systemd[1]: modprobe@fuse.service: Deactivated successfully.809vm-test-run-nixbot-gitlab> gitlab # [ 7.336129] systemd[1]: Finished Load Kernel Module fuse.810vm-test-run-nixbot-gitlab> gitlab # [ 7.337752] systemd[1]: Finished Load Kernel Modules.811vm-test-run-nixbot-gitlab> gitlab # [ 7.359893] systemd[1]: Queued start job for default target Multi-User System.812vm-test-run-nixbot-gitlab> gitlab # [ 7.362168] systemd[1]: Mounting FUSE Control File System...813vm-test-run-nixbot-gitlab> gitlab # [ 7.363293] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 7.367935] systemd[1]: Starting Firewall...814vm-test-run-nixbot-gitlab> gitlab # 815vm-test-run-nixbot-gitlab> gitlab # [ 7.372674] systemd[1]: systemd-journald.service: Deactivated successfully.816vm-test-run-nixbot-gitlab> gitlab # [ 7.374392] systemd-modules-load[806]: Using 3 probe threads817vm-test-run-nixbot-gitlab> gitlab # [ 7.375745] systemd-modules-load[806]: Module 'atkbd' is built in818vm-test-run-nixbot-gitlab> gitlab # [ 7.378110] systemd[1]: Starting Apply Kernel Variables...819vm-test-run-nixbot-gitlab> gitlab # [ 7.377946] systemd-modules-load[806]: Module 'loop' is built in820vm-test-run-nixbot-gitlab> gitlab # [ 7.380509] systemd-modules-load[806]: Inserted module 'tls'821vm-test-run-nixbot-gitlab> gitlab # [ 7.386892] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...822vm-test-run-nixbot-gitlab> gitlab # [ 7.387670] systemd[1]: Started Journal Service.823vm-test-run-nixbot-gitlab> gitlab # [ 7.390790] EXT4-fs (vda): re-mounted 91c97e3c-8f24-4d1f-ae7a-ef10c3ffb8f2.824vm-test-run-nixbot-gitlab> gitlab # [ 7.399339] systemd[1]: Finished Remount Root and Kernel File Systems.825vm-test-run-nixbot-gitlab> gitlab # [ 7.407586] systemd-oomd[808]: No swap; memory pressure usage will be degraded826vm-test-run-nixbot-gitlab> gitlab # [ 7.412451] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.827vm-test-run-nixbot-gitlab> gitlab # [ 7.420285] systemd[1]: Mounted FUSE Control File System.828vm-test-run-nixbot-gitlab> gitlab # [ 7.431693] systemd[1]: Starting Flush Journal to Persistent Storage...829vm-test-run-nixbot-gitlab> gitlab # [ 7.434321] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore830vm-test-run-nixbot-gitlab> gitlab # [ 7.440212] systemd[1]: Starting Load/Save OS Random Seed...831vm-test-run-nixbot-gitlab> gitlab # [ 7.441167] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki832vm-test-run-nixbot-gitlab> gitlab # [ 7.444341] systemd[1]: Finished Apply Kernel Variables.833vm-test-run-nixbot-gitlab> gitlab # [ 7.459268] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.834vm-test-run-nixbot-gitlab> gitlab # [ 7.463769] systemd[1]: Starting Create Static Device Nodes in /dev...835vm-test-run-nixbot-gitlab> gitlab # [ 7.472219] systemd-journald[805]: Received client request to flush runtime journal.836vm-test-run-nixbot-gitlab> gitlab # [ 7.500117] systemd[1]: Finished Load/Save OS Random Seed.837vm-test-run-nixbot-gitlab> gitlab # [ 7.501691] systemd[1]: Reached target First Boot Complete.838vm-test-run-nixbot-gitlab> gitlab # [ 7.503036] systemd[1]: Finished Create Static Device Nodes in /dev.839vm-test-run-nixbot-gitlab> gitlab # [ 7.504467] systemd[1]: Reached target Preparation for Local File Systems.840vm-test-run-nixbot-gitlab> gitlab # [ 7.505818] systemd[1]: Starting Rule-based Manager for Device Events and Files...841vm-test-run-nixbot-gitlab> gitlab # [ 7.507356] systemd[1]: Finished Flush Journal to Persistent Storage.842vm-test-run-nixbot-gitlab> gitlab # [ 7.587800] systemd-udevd[855]: Using default interface naming scheme 'v260'.843vm-test-run-nixbot-gitlab> gitlab # [ 7.649158] systemd[1]: Started Rule-based Manager for Device Events and Files.844vm-test-run-nixbot-gitlab> gitlab # [ 7.911989] systemd[1]: Finished Firewall.845vm-test-run-nixbot-gitlab> gitlab # [ 8.060663] systemd[1]: Mounting /run/wrappers...846vm-test-run-nixbot-gitlab> gitlab # [ 8.117315] systemd[1]: Mounted /run/wrappers.847vm-test-run-nixbot-gitlab> gitlab # [ 8.119657] systemd[1]: Reached target Local File Systems.848vm-test-run-nixbot-gitlab> gitlab # [ 8.123074] systemd[1]: Listening on Boot Loader Control Service Socket.849vm-test-run-nixbot-gitlab> gitlab # [ 8.128286] systemd[1]: Starting register-nix-paths.service...850vm-test-run-nixbot-gitlab> gitlab # [ 8.135339] systemd[1]: Starting Create SUID/SGID Wrappers...851vm-test-run-nixbot-gitlab> gitlab # [ 8.139559] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852vm-test-run-nixbot-gitlab> gitlab # [ 8.168561] systemd[1]: Starting Save Transient machine-id to Disk...853vm-test-run-nixbot-gitlab> gitlab # [ 8.186661] systemd[1]: Starting Create System Files and Directories...854vm-test-run-nixbot-gitlab> gitlab # [ 8.210138] systemd[1]: Finished Coldplug All udev Devices.855vm-test-run-nixbot-gitlab> gitlab # [ 8.222969] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.856vm-test-run-nixbot-gitlab> gitlab # [ 8.226380] systemd[1]: Finished Save Transient machine-id to Disk.857vm-test-run-nixbot-gitlab> gitlab # [ 8.272159] systemd[1]: Finished Create System Files and Directories.858vm-test-run-nixbot-gitlab> gitlab # [ 8.279214] systemd[1]: Starting Rebuild Journal Catalog...859vm-test-run-nixbot-gitlab> gitlab # [ 8.284763] systemd[1]: Starting Record System Boot/Shutdown in UTMP...860vm-test-run-nixbot-gitlab> gitlab # [ 8.332469] systemd[1]: Finished Record System Boot/Shutdown in UTMP.861vm-test-run-nixbot-gitlab> gitlab # [ 8.349220] systemd[1]: Finished Rebuild Journal Catalog.862vm-test-run-nixbot-gitlab> gitlab # [ 8.354332] systemd[1]: Starting Update is Completed...863vm-test-run-nixbot-gitlab> gitlab # [ 8.373506] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs864vm-test-run-nixbot-gitlab> gitlab # [ 8.393087] systemd[1]: Finished Update is Completed.865vm-test-run-nixbot-gitlab> gitlab # [ 8.434372] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.866vm-test-run-nixbot-gitlab> gitlab # [ 8.436162] systemd[1]: Finished Create SUID/SGID Wrappers.867vm-test-run-nixbot-gitlab> gitlab # [ 8.442001] systemd[1]: suid-sgid-wrappers.service: Consumed 220ms CPU time over 299ms wall clock time, 6.8M memory peak, 2.2M read from disk.868vm-test-run-nixbot-gitlab> gitlab # [ 8.450264] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.869vm-test-run-nixbot-gitlab> gitlab # [ 8.464227] (udev-worker)[908]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.870vm-test-run-nixbot-gitlab> gitlab # [ 8.467217] (udev-worker)[908]: Network interface NamePolicy= disabled on kernel command line.871vm-test-run-nixbot-gitlab> gitlab # [ 8.477158] (udev-worker)[1018]: Network interface NamePolicy= disabled on kernel command line.872vm-test-run-nixbot-gitlab> gitlab # [ 8.480412] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.873vm-test-run-nixbot-gitlab> gitlab # [ 8.493845] systemd[1]: Condition check resulted in Virtio network device being skipped.874vm-test-run-nixbot-gitlab> gitlab # [ 8.495633] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore875vm-test-run-nixbot-gitlab> gitlab # [ 8.499539] systemd[1]: Starting Create SUID/SGID Wrappers...876vm-test-run-nixbot-gitlab> gitlab # [ 8.501121] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.877vm-test-run-nixbot-gitlab> gitlab # [ 8.502696] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67878vm-test-run-nixbot-gitlab> gitlab # [ 8.505284] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore879vm-test-run-nixbot-gitlab> gitlab # [ 8.507713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki880vm-test-run-nixbot-gitlab> gitlab # [ 8.510173] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki881vm-test-run-nixbot-gitlab> gitlab # [ 8.660023] mousedev: PS/2 mouse device common for all mice882vm-test-run-nixbot-gitlab> gitlab # [ 8.714032] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.883vm-test-run-nixbot-gitlab> gitlab # [ 8.715895] systemd[1]: Finished Create SUID/SGID Wrappers.884vm-test-run-nixbot-gitlab> gitlab # [ 8.752681] systemd[1]: Finished register-nix-paths.service.885vm-test-run-nixbot-gitlab> gitlab # [ 8.754275] systemd[1]: Reached target System Initialization.886vm-test-run-nixbot-gitlab> gitlab # [ 8.755384] systemd[1]: Started Discard unused filesystem blocks once a week.887vm-test-run-nixbot-gitlab> gitlab # [ 8.757518] systemd[1]: Started logrotate.timer.888vm-test-run-nixbot-gitlab> gitlab # [ 8.758560] systemd[1]: Started Daily Cleanup of Temporary Directories.889vm-test-run-nixbot-gitlab> gitlab # [ 8.759749] systemd[1]: Reached target Timer Units.890vm-test-run-nixbot-gitlab> gitlab # [ 8.762125] systemd[1]: Listening on D-Bus System Message Bus Socket.891vm-test-run-nixbot-gitlab> gitlab # [ 8.763449] systemd[1]: Listening on Nix Daemon Socket.892vm-test-run-nixbot-gitlab> gitlab # [ 8.773593] systemd[1]: Starting nixbot CI service socket...893vm-test-run-nixbot-gitlab> gitlab # [ 8.775047] systemd[1]: Listening on Hostname Service Socket.894vm-test-run-nixbot-gitlab> gitlab # [ 8.779394] systemd[1]: Listening on nixbot CI service socket.895vm-test-run-nixbot-gitlab> gitlab # [ 8.780385] systemd[1]: Reached target Socket Units.896vm-test-run-nixbot-gitlab> gitlab # [ 8.781135] systemd[1]: Starting D-Bus System Message Bus...897vm-test-run-nixbot-gitlab> gitlab # [ 8.821495] dbus-broker-launch[1124]: Looking up NSS user entry for 'systemd-timesync'...898vm-test-run-nixbot-gitlab> gitlab # [ 8.825652] dbus-broker-launch[1124]: NSS returned no entry for 'systemd-timesync'899vm-test-run-nixbot-gitlab> gitlab # [ 8.827793] dbus-broker-launch[1124]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"900vm-test-run-nixbot-gitlab> gitlab # [ 8.832613] systemd[1]: Started D-Bus System Message Bus.901vm-test-run-nixbot-gitlab> gitlab # [ 8.842077] dbus-broker-launch[1124]: Ready902vm-test-run-nixbot-gitlab> gitlab # [ 8.843318] systemd[1]: Reached target Basic System.903vm-test-run-nixbot-gitlab> gitlab # [ 8.845837] systemd[1]: Started backdoor.service.904vm-test-run-nixbot-gitlab> gitlab # [ 8.848960] systemd[1]: Starting gitlab-config.service...905vm-test-run-nixbot-gitlab> gitlab # [ 8.854273] systemd[1]: Starting Import lastlog data into lastlog2 database...906vm-test-run-nixbot-gitlab> gitlab # [ 8.867982] systemd[1]: Starting Logrotate configuration check...907vm-test-run-nixbot-gitlab> gitlab # [ 8.876086] systemd[1]: Starting Name Service Cache Daemon (nsncd)...908vm-test-run-nixbot-gitlab> gitlab # [ 8.882114] systemd[1]: Starting Post-Boot Actions...909vm-test-run-nixbot-gitlab> gitlab # [ 8.883290] logrotate[1129]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.910vm-test-run-nixbot-gitlab> gitlab # connecting to host...911vm-test-run-nixbot-gitlab> gitlab # [ 8.885847] logrotate[1129]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf912vm-test-run-nixbot-gitlab> gitlab # [ 8.887280] logrotate[1129]: note: 'monthly' overrides previously specified 'weekly'913vm-test-run-nixbot-gitlab> gitlab # [ 8.889986] logrotate[1129]: Reading state from file: /var/lib/logrotate.status914vm-test-run-nixbot-gitlab> gitlab # [ 8.891540] logrotate[1129]: state file /var/lib/logrotate.status does not exist915vm-test-run-nixbot-gitlab> gitlab # [ 8.895083] logrotate[1129]: Allocating hash table for state file, size 64 entries916vm-test-run-nixbot-gitlab> gitlab # [ 8.896240] logrotate[1129]: Handling 4 logs917vm-test-run-nixbot-gitlab> gitlab # [ 8.896856] logrotate[1129]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed918vm-test-run-nixbot-gitlab> gitlab # [ 8.898980] logrotate[1129]: considering log /var/log/btmp919vm-test-run-nixbot-gitlab> gitlab # [ 8.899721] logrotate[1129]: Creating new state920vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'921vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell922vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.35 seconds)923vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.35 seconds)924vm-test-run-nixbot-gitlab> gitlab # [ 8.904235] logrotate[1129]: Now: 2026-06-29 21:54925vm-test-run-nixbot-gitlab> gitlab # [ 8.905006] logrotate[1129]: Last rotated at 2026-06-29 21:00926vm-test-run-nixbot-gitlab> gitlab # [ 8.905765] logrotate[1129]: log does not need rotating (log has already been rotated)927vm-test-run-nixbot-gitlab> gitlab # [ 8.906780] logrotate[1129]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed928vm-test-run-nixbot-gitlab> gitlab # [ 8.909630] logrotate[1129]: considering log /var/log/wtmp929vm-test-run-nixbot-gitlab> gitlab # [ 8.910378] logrotate[1129]: Creating new state930vm-test-run-nixbot-gitlab> gitlab # [ 8.911076] logrotate[1129]: Now: 2026-06-29 21:54931vm-test-run-nixbot-gitlab> gitlab # [ 8.911746] logrotate[1129]: Last rotated at 2026-06-29 21:00932vm-test-run-nixbot-gitlab> gitlab # [ 8.916991] logrotate[1129]: log does not need rotating (log has already been rotated)933vm-test-run-nixbot-gitlab> gitlab # [ 8.918106] logrotate[1129]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed934vm-test-run-nixbot-gitlab> gitlab # [ 8.919934] logrotate[1129]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1129)935vm-test-run-nixbot-gitlab> gitlab # [ 8.923185] logrotate[1129]: considering log /var/gitlab/state/log/*.log936vm-test-run-nixbot-gitlab> gitlab # [ 8.924228] logrotate[1129]: log /var/gitlab/state/log/*.log does not exist -- skipping937vm-test-run-nixbot-gitlab> gitlab # [ 8.925388] logrotate[1129]: Creating new state938vm-test-run-nixbot-gitlab> gitlab # [ 8.926117] logrotate[1129]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1129)939vm-test-run-nixbot-gitlab> gitlab # [ 8.931168] logrotate[1129]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed940vm-test-run-nixbot-gitlab> gitlab # [ 8.933569] logrotate[1129]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1129)941vm-test-run-nixbot-gitlab> gitlab # [ 8.934761] logrotate[1129]: considering log /var/log/nginx/*.log942vm-test-run-nixbot-gitlab> gitlab # [ 8.935592] logrotate[1129]: log /var/log/nginx/*.log does not exist -- skipping943vm-test-run-nixbot-gitlab> gitlab # [ 8.938553] logrotate[1129]: Creating new state944vm-test-run-nixbot-gitlab> gitlab # [ 8.939251] logrotate[1129]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1129)945vm-test-run-nixbot-gitlab> gitlab # [ 8.940869] systemd[1]: Started Reset console on configuration changes.946vm-test-run-nixbot-gitlab> gitlab # [ 8.941833] systemd[1]: Starting resolvconf update...947vm-test-run-nixbot-gitlab> gitlab # [ 8.942832] systemd[1]: Finished Logrotate configuration check.948vm-test-run-nixbot-gitlab> gitlab # [ 8.944665] nsncd[1130]: Jun 29 21:54:36.854 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"949vm-test-run-nixbot-gitlab> gitlab # [ 8.947084] systemd[1]: Finished Post-Boot Actions.950vm-test-run-nixbot-gitlab> gitlab # [ 8.947970] systemd[1]: Finished Import lastlog data into lastlog2 database.951vm-test-run-nixbot-gitlab> gitlab # [ 8.949350] systemd[1]: Started Name Service Cache Daemon (nsncd).952vm-test-run-nixbot-gitlab> gitlab # [ 8.950380] systemd[1]: Reached target Host and Network Name Lookups.953vm-test-run-nixbot-gitlab> gitlab # [ 8.952902] systemd[1]: Reached target User and Group Name Lookups.954vm-test-run-nixbot-gitlab> gitlab # [ 8.954887] systemd[1]: Starting User Login Management...955vm-test-run-nixbot-gitlab> gitlab # [ 9.007700] systemd-logind[1169]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)956vm-test-run-nixbot-gitlab> gitlab # [ 9.009726] systemd-logind[1169]: Watching system buttons on /dev/input/event0 (gpio-keys)957vm-test-run-nixbot-gitlab> gitlab # [ 9.010862] systemd-logind[1169]: New seat seat0.958vm-test-run-nixbot-gitlab> gitlab # [ 9.015694] systemd[1]: Started User Login Management.959vm-test-run-nixbot-gitlab> gitlab # [ 9.018004] systemd[1]: Starting linger-users.service...960vm-test-run-nixbot-gitlab> gitlab # [ 9.030274] systemd[1]: Stopped target Host and Network Name Lookups.961vm-test-run-nixbot-gitlab> gitlab # [ 9.032889] systemd[1]: Stopping Host and Network Name Lookups...962vm-test-run-nixbot-gitlab> gitlab # [ 9.033961] systemd[1]: Stopped target User and Group Name Lookups.963vm-test-run-nixbot-gitlab> gitlab # [ 9.034834] systemd[1]: Stopping User and Group Name Lookups...964vm-test-run-nixbot-gitlab> gitlab # [ 9.035681] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...965vm-test-run-nixbot-gitlab> gitlab # [ 9.038729] systemd[1]: nscd.service: Deactivated successfully.966vm-test-run-nixbot-gitlab> gitlab # [ 9.039708] systemd[1]: Stopped Name Service Cache Daemon (nsncd).967vm-test-run-nixbot-gitlab> gitlab # [ 9.043335] systemd[1]: nscd.service: Consumed 23ms CPU time over 156ms wall clock time, 7.9M memory peak, 5.9M read from disk.968vm-test-run-nixbot-gitlab> gitlab # [ 9.050930] systemd[1]: Starting Name Service Cache Daemon (nsncd)...969vm-test-run-nixbot-gitlab> gitlab # [ 9.053672] systemd[1]: linger-users.service: Deactivated successfully.970vm-test-run-nixbot-gitlab> gitlab # [ 9.054878] systemd[1]: Finished linger-users.service.971vm-test-run-nixbot-gitlab> gitlab # [ 9.057868] systemd[1]: Finished resolvconf update.972vm-test-run-nixbot-gitlab> gitlab # [ 9.061196] systemd[1]: Reached target Preparation for Network.973vm-test-run-nixbot-gitlab> gitlab # [ 9.066163] systemd[1]: Starting DHCP Client...974vm-test-run-nixbot-gitlab> gitlab # [ 9.067595] systemd[1]: Starting Address configuration of eth1...975vm-test-run-nixbot-gitlab> gitlab # [ 9.073387] systemd[1]: Starting Extra networking commands....976vm-test-run-nixbot-gitlab> gitlab # [ 9.080854] systemd[1]: Started Name Service Cache Daemon (nsncd).977vm-test-run-nixbot-gitlab> gitlab # [ 9.084830] nsncd[1224]: Jun 29 21:54:36.995 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"978vm-test-run-nixbot-gitlab> gitlab # [ 9.087013] systemd[1]: Reached target Host and Network Name Lookups.979vm-test-run-nixbot-gitlab> gitlab # [ 9.089890] systemd[1]: Reached target User and Group Name Lookups.980vm-test-run-nixbot-gitlab> gitlab # [ 9.117239] network-addresses-eth1-start[1232]: adding address 192.168.1.1/24... done981vm-test-run-nixbot-gitlab> gitlab # [ 9.120538] network-addresses-eth1-start[1232]: adding address 2001:db8:1::1/64... done982vm-test-run-nixbot-gitlab> gitlab # [ 9.134014] systemd[1]: Finished Address configuration of eth1.983vm-test-run-nixbot-gitlab> gitlab # [ 9.176515] dhcpcd[1249]: dhcpcd-10.3.2 starting984vm-test-run-nixbot-gitlab> gitlab # [ 9.179984] dhcpcd[1293]: dev: loaded udev985vm-test-run-nixbot-gitlab> gitlab # [ 9.195607] 8021q: 802.1Q VLAN Support v1.8986vm-test-run-nixbot-gitlab> gitlab # [ 9.196046] 8021q: adding VLAN 0 to HW filter on device eth1987vm-test-run-nixbot-gitlab> gitlab # [ 9.194805] systemd[1]: Finished Extra networking commands..988vm-test-run-nixbot-gitlab> gitlab # [ 9.197028] systemd[1]: Reached target Network.989vm-test-run-nixbot-gitlab> gitlab # [ 9.203557] systemd[1]: Starting gitlab-workhorse.service...990vm-test-run-nixbot-gitlab> gitlab # [ 9.212565] systemd[1]: Starting Nginx Web Server...991vm-test-run-nixbot-gitlab> gitlab # [ 9.224499] systemd[1]: Starting PostgreSQL Server...992vm-test-run-nixbot-gitlab> gitlab # [ 9.232618] systemd[1]: Starting Redis Server - redis-gitlab...993vm-test-run-nixbot-gitlab> gitlab # [ 9.244181] systemd[1]: Starting Permit User Sessions...994vm-test-run-nixbot-gitlab> gitlab # [ 9.249187] cfg80211: Loading compiled-in X.509 certificates for regulatory database995vm-test-run-nixbot-gitlab> gitlab # [ 9.255054] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'996vm-test-run-nixbot-gitlab> gitlab # [ 9.255511] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'997vm-test-run-nixbot-gitlab> gitlab # [ 9.257027] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2998vm-test-run-nixbot-gitlab> gitlab # [ 9.257344] cfg80211: failed to load regulatory.db999vm-test-run-nixbot-gitlab> gitlab # [ 9.277146] systemd[1]: Finished Permit User Sessions.1000vm-test-run-nixbot-gitlab> gitlab # [ 9.285797] systemd[1]: Started Getty on tty1.1001vm-test-run-nixbot-gitlab> gitlab # [ 9.286540] systemd[1]: Reached target Login Prompts.1002vm-test-run-nixbot-gitlab> gitlab # [ 9.371442] 8021q: adding VLAN 0 to HW filter on device eth01003vm-test-run-nixbot-gitlab> gitlab # [ 9.371747] dhcpcd[1293]: eth0: waiting for carrier1004vm-test-run-nixbot-gitlab> gitlab # [ 9.373175] dhcpcd[1293]: eth0: carrier acquired1005vm-test-run-nixbot-gitlab> gitlab # [ 9.380140] dhcpcd[1293]: DUID 00:01:00:01:31:d5:a6:1d:52:54:00:12:34:561006vm-test-run-nixbot-gitlab> gitlab # [ 9.381366] dhcpcd[1293]: eth0: IAID 00:12:34:561007vm-test-run-nixbot-gitlab> gitlab # [ 9.382081] dhcpcd[1293]: eth0: adding address fe80::5054:ff:fe12:34561008vm-test-run-nixbot-gitlab> gitlab # [ 9.416505] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input31009vm-test-run-nixbot-gitlab> gitlab # [ 9.423277] nginx-pre-start[1331]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok1010vm-test-run-nixbot-gitlab> gitlab # [ 9.428744] nginx-pre-start[1331]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful1011vm-test-run-nixbot-gitlab> gitlab # [ 9.435876] systemd[1]: Started Nginx Web Server.1012vm-test-run-nixbot-gitlab> gitlab # [ 9.451733] postgresql-pre-start[1330]: The files belonging to this database system will be owned by user "postgres".1013vm-test-run-nixbot-gitlab> gitlab # [ 9.453723] postgresql-pre-start[1330]: This user must also own the server process.1014vm-test-run-nixbot-gitlab> gitlab # [ 9.456444] postgresql-pre-start[1330]: The database cluster will be initialized with locale "en_US.UTF-8".1015vm-test-run-nixbot-gitlab> gitlab # [ 9.457740] postgresql-pre-start[1330]: The default database encoding has accordingly been set to "UTF8".1016vm-test-run-nixbot-gitlab> gitlab # [ 9.459026] postgresql-pre-start[1330]: The default text search configuration will be set to "english".1017vm-test-run-nixbot-gitlab> gitlab # [ 9.464337] postgresql-pre-start[1330]: Data page checksums are disabled.1018vm-test-run-nixbot-gitlab> gitlab # [ 9.465528] postgresql-pre-start[1330]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1019vm-test-run-nixbot-gitlab> gitlab # [ 9.466875] postgresql-pre-start[1330]: creating subdirectories ... ok1020vm-test-run-nixbot-gitlab> gitlab # [ 9.468139] postgresql-pre-start[1330]: selecting dynamic shared memory implementation ... posix1021vm-test-run-nixbot-gitlab> gitlab # [ 9.476979] redis-server[1336]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1022vm-test-run-nixbot-gitlab> gitlab # [ 9.479581] redis-server[1336]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1023vm-test-run-nixbot-gitlab> gitlab # [ 9.488317] redis-server[1336]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1336, just started1024vm-test-run-nixbot-gitlab> gitlab # [ 9.489895] systemd[1]: Started Redis Server - redis-gitlab.1025vm-test-run-nixbot-gitlab> gitlab # [ 9.490767] redis-server[1336]: Configuration loaded1026vm-test-run-nixbot-gitlab> gitlab # [ 9.491707] redis[1336]: monotonic clock: ARM CNTVCT @ 1000 ticks/us1027vm-test-run-nixbot-gitlab> gitlab # [ 9.496239] redis[1336]: Running mode=standalone, port=0.1028vm-test-run-nixbot-gitlab> gitlab # [ 9.497257] redis[1336]: Server initialized1029vm-test-run-nixbot-gitlab> gitlab # [ 9.498023] redis[1336]: Ready to accept connections unix1030vm-test-run-nixbot-gitlab> gitlab # [ 9.498946] redis[1336]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1031vm-test-run-nixbot-gitlab> gitlab # [ 9.545482] postgresql-pre-start[1330]: selecting default "max_connections" ... 1001032vm-test-run-nixbot-gitlab> gitlab # [ 9.590969] postgresql-pre-start[1330]: selecting default "shared_buffers" ... 128MB1033vm-test-run-nixbot-gitlab> gitlab # [ 9.719215] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1034vm-test-run-nixbot-gitlab> gitlab # [ 9.735804] systemd[1]: Starting Virtual Console Setup...1035vm-test-run-nixbot-gitlab> gitlab # [ 9.757642] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.1036vm-test-run-nixbot-gitlab> gitlab # [ 9.759358] systemd[1]: Stopped Virtual Console Setup.1037vm-test-run-nixbot-gitlab> gitlab # [ 9.764652] systemd[1]: Starting Virtual Console Setup...1038vm-test-run-nixbot-gitlab> gitlab # [ 9.794021] systemd-logind[1169]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)1039vm-test-run-nixbot-gitlab> gitlab # [ 9.829484] systemd-vconsole-setup[1365]: Configuration of first virtual console was skipped, ignoring remaining ones.1040vm-test-run-nixbot-gitlab> gitlab # [ 9.833078] systemd[1]: Finished Virtual Console Setup.1041vm-test-run-nixbot-gitlab> gitlab # [ 9.836907] systemd[1]: Started gitlab-workhorse.service.1042vm-test-run-nixbot-gitlab> gitlab # [ 10.136947] postgresql-pre-start[1330]: selecting default time zone ... UTC1043vm-test-run-nixbot-gitlab> gitlab # [ 10.139737] postgresql-pre-start[1330]: creating configuration files ... ok1044vm-test-run-nixbot-gitlab> gitlab # [ 10.187070] gitlab-workhorse[1369]: time="2026-06-29T21:54:38Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41045vm-test-run-nixbot-gitlab> gitlab # [ 10.190785] gitlab-workhorse[1369]: time="2026-06-29T21:54:38Z" level=info msg="Using redis/go-redis"1046vm-test-run-nixbot-gitlab> gitlab # [ 10.193533] gitlab-workhorse[1369]: time="2026-06-29T21:54:38Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1047vm-test-run-nixbot-gitlab> gitlab # [ 10.360382] postgresql-pre-start[1330]: running bootstrap script ... ok1048vm-test-run-nixbot-gitlab> gitlab # [ 10.487114] dhcpcd[1293]: eth0: soliciting a DHCP lease1049vm-test-run-nixbot-gitlab> gitlab # [ 10.496854] dhcpcd[1293]: eth0: offered 10.0.2.15 from 10.0.2.21050vm-test-run-nixbot-gitlab> gitlab # [ 10.512459] dhcpcd[1293]: eth0: probing address 10.0.2.15/241051vm-test-run-nixbot-gitlab> gitlab # [ 11.055194] postgresql-pre-start[1330]: performing post-bootstrap initialization ... ok1052vm-test-run-nixbot-gitlab> gitlab # [ 11.358034] postgresql-pre-start[1330]: syncing data to disk ... ok1053vm-test-run-nixbot-gitlab> gitlab # [ 11.360132] postgresql-pre-start[1330]: initdb: warning: enabling "trust" authentication for local connections1054vm-test-run-nixbot-gitlab> gitlab # [ 11.363034] postgresql-pre-start[1330]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1055vm-test-run-nixbot-gitlab> gitlab # [ 11.367892] postgresql-pre-start[1330]: Success. You can now start the database server using:1056vm-test-run-nixbot-gitlab> gitlab # [ 11.370503] postgresql-pre-start[1330]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1057vm-test-run-nixbot-gitlab> gitlab # [ 11.581808] postgres[1403]: [1403] LOG: starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit1058vm-test-run-nixbot-gitlab> gitlab # [ 11.587696] postgres[1403]: [1403] LOG: listening on IPv6 address "::1", port 54321059vm-test-run-nixbot-gitlab> gitlab # [ 11.592528] postgres[1403]: [1403] LOG: listening on IPv4 address "127.0.0.1", port 54321060vm-test-run-nixbot-gitlab> gitlab # [ 11.596115] postgres[1403]: [1403] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1061vm-test-run-nixbot-gitlab> gitlab # [ 11.601667] postgres[1414]: [1414] LOG: database system was shut down at 2026-06-29 21:54:38 GMT1062vm-test-run-nixbot-gitlab> gitlab # [ 11.605933] postgres[1403]: [1403] LOG: database system is ready to accept connections1063vm-test-run-nixbot-gitlab> gitlab # [ 11.610051] systemd[1]: Started PostgreSQL Server.1064vm-test-run-nixbot-gitlab> gitlab # [ 11.614243] systemd[1]: Starting PostgreSQL Setup Scripts...1065vm-test-run-nixbot-gitlab> gitlab # [ 11.815465] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1411]: mkdir -p /var/gitlab/state/home/.ssh: OK1066vm-test-run-nixbot-gitlab> gitlab # [ 11.819636] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1411]: chmod 700 /var/gitlab/state/home/.ssh: OK1067vm-test-run-nixbot-gitlab> gitlab # [ 11.873345] postgresql-setup-start[1426]: CREATE DATABASE1068vm-test-run-nixbot-gitlab> gitlab # [ 11.896514] systemd[1]: Finished gitlab-config.service.1069vm-test-run-nixbot-gitlab> gitlab # [ 11.903454] systemd[1]: Started gitaly.service.1070vm-test-run-nixbot-gitlab> gitlab # [ 11.920914] postgresql-setup-start[1442]: CREATE ROLE1071vm-test-run-nixbot-gitlab> gitlab # [ 11.943351] postgresql-setup-start[1446]: ALTER DATABASE1072vm-test-run-nixbot-gitlab> gitlab # [ 11.993786] postgresql-setup-start[1452]: CREATE ROLE1073vm-test-run-nixbot-gitlab> gitlab # [ 11.997273] systemd[1]: Finished PostgreSQL Setup Scripts.1074vm-test-run-nixbot-gitlab> gitlab # [ 12.000001] systemd[1]: Reached target PostgreSQL.1075vm-test-run-nixbot-gitlab> gitlab # [ 12.005291] systemd[1]: Starting gitlab-db-config.service...1076vm-test-run-nixbot-gitlab> gitlab # [ 12.012310] systemd[1]: Starting gitlab-postgresql.service...1077vm-test-run-nixbot-gitlab> gitlab # [ 12.090319] dhcpcd[1293]: eth0: soliciting an IPv6 router1078vm-test-run-nixbot-gitlab> gitlab # [ 12.092379] dhcpcd[1293]: eth0: Router Advertisement from fe80::21079vm-test-run-nixbot-gitlab> gitlab # [ 12.094280] dhcpcd[1293]: eth0: adding address fec0::5054:ff:fe12:3456/641080vm-test-run-nixbot-gitlab> gitlab # [ 12.096575] dhcpcd[1293]: eth0: adding route to fec0::/641081vm-test-run-nixbot-gitlab> gitlab # [ 12.098381] dhcpcd[1293]: eth0: adding default route via fe80::21082vm-test-run-nixbot-gitlab> gitlab # [ 12.209373] gitlab-postgresql-start[1462]: CREATE DATABASE1083vm-test-run-nixbot-gitlab> gitlab # [ 12.280454] gitlab-postgresql-start[1477]: CREATE EXTENSION1084vm-test-run-nixbot-gitlab> gitlab # [ 12.338058] gitlab-postgresql-start[1484]: CREATE EXTENSION1085vm-test-run-nixbot-gitlab> gitlab # [ 12.342791] systemd[1]: Finished gitlab-postgresql.service.1086vm-test-run-nixbot-gitlab> gitlab # [ 12.586005] gitaly[1444]: time="2026-06-29T21:54:40.500Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=14441087vm-test-run-nixbot-gitlab> gitlab # [ 12.594503] gitaly[1444]: time="2026-06-29T21:54:40.500Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=14441088vm-test-run-nixbot-gitlab> gitlab # [ 12.598421] gitaly[1444]: time="2026-06-29T21:54:40.500Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=14441089vm-test-run-nixbot-gitlab> gitlab # [ 12.603720] gitaly[1444]: time="2026-06-29T21:54:40.500Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=14441090vm-test-run-nixbot-gitlab> gitlab # [ 12.608540] gitaly[1444]: time="2026-06-29T21:54:40.500Z" level=info msg="Starting Gitaly" pid=1444 version=18.11.41091vm-test-run-nixbot-gitlab> gitlab # [ 12.834771] gitaly[1444]: time="2026-06-29T21:54:40.750Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=246 pid=14441092vm-test-run-nixbot-gitlab> gitlab # [ 12.836906] gitaly[1444]: time="2026-06-29T21:54:40.750Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=14441093vm-test-run-nixbot-gitlab> gitlab # [ 12.839250] gitaly[1444]: time="2026-06-29T21:54:40.750Z" level=info msg="finished initializing command factory" duration_ms=0 pid=14441094vm-test-run-nixbot-gitlab> gitlab # [ 12.841519] gitaly[1444]: time="2026-06-29T21:54:40.750Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1444/git-exec-3104831686.d/git pid=14441095vm-test-run-nixbot-gitlab> gitlab # [ 12.844851] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="finished detecting git version" duration_ms=9 pid=14441096vm-test-run-nixbot-gitlab> gitlab # [ 12.846856] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="using Git version" pid=1444 version=2.53.g1097vm-test-run-nixbot-gitlab> gitlab # [ 12.849087] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="clearing disk cache object folder" pid=1444 storage=default1098vm-test-run-nixbot-gitlab> gitlab # [ 12.851128] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3617273659 pid=1444 storage=default1099vm-test-run-nixbot-gitlab> gitlab # [ 12.854177] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3617273659 pid=1444 storage=default1100vm-test-run-nixbot-gitlab> gitlab # [ 12.857536] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=14441101vm-test-run-nixbot-gitlab> gitlab # [ 12.860795] gitaly[1444]: time="2026-06-29T21:54:40.760Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=14441102vm-test-run-nixbot-gitlab> gitlab # [ 12.863739] gitaly[1444]: time="2026-06-29T21:54:40.761Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3617273659 clear_duration_ms=0 pid=1444 storage=default1103vm-test-run-nixbot-gitlab> gitlab # [ 12.867022] gitaly[1444]: time="2026-06-29T21:54:40.775Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=14441104vm-test-run-nixbot-gitlab> gitlab # [ 12.869660] gitaly[1444]: time="2026-06-29T21:54:40.775Z" level=info msg="counting repositories" pid=14441105vm-test-run-nixbot-gitlab> gitlab # [ 12.871216] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="starting to count repositories" pid=1444 storage_path=/var/gitlab/state/repositories1106vm-test-run-nixbot-gitlab> gitlab # [ 12.873570] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="finished tempdir cleaner walk" pid=1444 storage=default time_ms=01107vm-test-run-nixbot-gitlab> gitlab # [ 12.875471] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1444 schema=unix1108vm-test-run-nixbot-gitlab> gitlab # [ 12.877914] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1444 storage_path=/var/gitlab/state/repositories1109vm-test-run-nixbot-gitlab> gitlab # [ 12.880112] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="completed counting all repositories" counting_duration=0 pid=14441110vm-test-run-nixbot-gitlab> gitlab # [ 12.882257] gitaly[1444]: time="2026-06-29T21:54:40.776Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1444/sock.d/intern pid=1444 schema=unix1111vm-test-run-nixbot-gitlab> gitlab # [ 12.885013] gitaly[1444]: time="2026-06-29T21:54:40.777Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=14441112vm-test-run-nixbot-gitlab> gitlab # [ 12.886969] gitaly[1444]: time="2026-06-29T21:54:40.777Z" level=info msg="Started Gitaly" duration_ms=277 pid=14441113vm-test-run-nixbot-gitlab> gitlab # [ 12.888771] gitaly[1444]: time="2026-06-29T21:54:40.778Z" level=info msg="maintenance: daily scheduled" pid=1444 scheduled="2026-06-30 12:00:00 +0000 UTC"1114vm-test-run-nixbot-gitlab> gitlab # [ 13.298449] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1115vm-test-run-nixbot-gitlab> gitlab # [ 13.302251] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1116vm-test-run-nixbot-gitlab> gitlab # [ 13.317403] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1117vm-test-run-nixbot-gitlab> gitlab # [ 15.345636] dhcpcd[1293]: eth0: leased 10.0.2.15 for 86400 seconds1118vm-test-run-nixbot-gitlab> gitlab # [ 15.346056] dhcpcd[1293]: eth0: adding route to 10.0.2.0/241119vm-test-run-nixbot-gitlab> gitlab # [ 15.346261] dhcpcd[1293]: eth0: adding default route via 10.0.2.21120vm-test-run-nixbot-gitlab> gitlab # [ 15.565917] systemd[1]: Started DHCP Client.1121vm-test-run-nixbot-gitlab> gitlab # [ 15.566677] systemd[1]: Reached target Network is Online.1122vm-test-run-nixbot-gitlab> gitlab # [ 16.288298] gitaly[1444]: time="2026-06-29T21:54:44.202Z" level=info msg="License database preloaded" duration_ms=3701 pid=14441123vm-test-run-nixbot-gitlab> gitlab # [ 55.924852] postgres[1918]: [1918] ERROR: relation "feature_gates" does not exist at character 5231124vm-test-run-nixbot-gitlab> gitlab # [ 55.925500] postgres[1918]: [1918] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1125vm-test-run-nixbot-gitlab> gitlab # [ 55.925617] postgres[1918]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1126vm-test-run-nixbot-gitlab> gitlab # [ 55.925748] postgres[1918]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1127vm-test-run-nixbot-gitlab> gitlab # [ 55.925909] postgres[1918]: attidentity AS identity,1128vm-test-run-nixbot-gitlab> gitlab # [ 55.926023] postgres[1918]: attgenerated as attgenerated1129vm-test-run-nixbot-gitlab> gitlab # [ 55.926130] postgres[1918]: FROM pg_attribute a1130vm-test-run-nixbot-gitlab> gitlab # [ 55.926237] postgres[1918]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1131vm-test-run-nixbot-gitlab> gitlab # [ 55.926353] postgres[1918]: LEFT JOIN pg_type t ON a.atttypid = t.oid1132vm-test-run-nixbot-gitlab> gitlab # [ 55.926486] postgres[1918]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1133vm-test-run-nixbot-gitlab> gitlab # [ 55.926604] postgres[1918]: WHERE a.attrelid = '"feature_gates"'::regclass1134vm-test-run-nixbot-gitlab> gitlab # [ 55.926721] postgres[1918]: AND a.attnum > 0 AND NOT a.attisdropped1135vm-test-run-nixbot-gitlab> gitlab # [ 55.926828] postgres[1918]: ORDER BY a.attnum1136vm-test-run-nixbot-gitlab> gitlab # [ 55.926937] postgres[1918]:1137vm-test-run-nixbot-gitlab> gitlab # [ 56.251347] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1457]: W, [2026-06-29T21:55:24.165451 #1457] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1138vm-test-run-nixbot-gitlab> gitlab # [ 60.272164] gitlab-workhorse[1369]: time="2026-06-29T21:55:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1139vm-test-run-nixbot-gitlab> gitlab # [ 64.138923] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[2000]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1140vm-test-run-nixbot-gitlab> gitlab # [ 64.139371] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[2000]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1141vm-test-run-nixbot-gitlab> gitlab # [ 70.273701] gitlab-workhorse[1369]: time="2026-06-29T21:55:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1142vm-test-run-nixbot-gitlab> gitlab # [ 80.282310] gitlab-workhorse[1369]: time="2026-06-29T21:55:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143vm-test-run-nixbot-gitlab> gitlab # [ 88.016995] postgres[2085]: [2085] ERROR: relation "table_without_models" does not exist1144vm-test-run-nixbot-gitlab> gitlab # [ 88.020587] postgres[2085]: [2085] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1145vm-test-run-nixbot-gitlab> gitlab # [ 90.292490] gitlab-workhorse[1369]: time="2026-06-29T21:55:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146vm-test-run-nixbot-gitlab> gitlab # [ 100.302563] gitlab-workhorse[1369]: time="2026-06-29T21:56:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147vm-test-run-nixbot-gitlab> gitlab # [ 110.312468] gitlab-workhorse[1369]: time="2026-06-29T21:56:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1148vm-test-run-nixbot-gitlab> gitlab # [ 119.157558] systemd[1]: Finished gitlab-db-config.service.1149vm-test-run-nixbot-gitlab> gitlab # [ 119.163796] systemd[1]: Started gitlab-sidekiq.service.1150vm-test-run-nixbot-gitlab> gitlab # [ 119.170117] systemd[1]: Starting gitlab.service...1151vm-test-run-nixbot-gitlab> gitlab # [ 119.684271] bundle[2469]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1152vm-test-run-nixbot-gitlab> gitlab # [ 119.690674] bundle[2469]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1153vm-test-run-nixbot-gitlab> gitlab # [ 119.697417] bundle[2469]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1154vm-test-run-nixbot-gitlab> gitlab # [ 120.322721] gitlab-workhorse[1369]: time="2026-06-29T21:56:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1155vm-test-run-nixbot-gitlab> gitlab # [ 125.521768] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.400Z","pid":2469,"message":"* Enabling systemd notification integration"}1156vm-test-run-nixbot-gitlab> gitlab # [ 125.522184] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"Puma starting in cluster mode..."}1157vm-test-run-nixbot-gitlab> gitlab # [ 125.522451] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1158vm-test-run-nixbot-gitlab> gitlab # [ 125.522627] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1159vm-test-run-nixbot-gitlab> gitlab # [ 125.522805] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Min threads: 0"}1160vm-test-run-nixbot-gitlab> gitlab # [ 125.522969] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Max threads: 4"}1161vm-test-run-nixbot-gitlab> gitlab # [ 125.523135] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Environment: production"}1162vm-test-run-nixbot-gitlab> gitlab # [ 125.523297] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.435Z","pid":2469,"message":"* Master PID: 2469"}1163vm-test-run-nixbot-gitlab> gitlab # [ 125.523495] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.436Z","pid":2469,"message":"* Workers: 2"}1164vm-test-run-nixbot-gitlab> gitlab # [ 125.523636] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.436Z","pid":2469,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1165vm-test-run-nixbot-gitlab> gitlab # [ 125.523772] bundle[2469]: {"timestamp":"2026-06-29T21:56:33.436Z","pid":2469,"message":"* Preloading application"}1166vm-test-run-nixbot-gitlab> gitlab # [ 125.652141] sidekiq-cluster[2468]: {"severity":"INFO","time":"2026-06-29T21:56:33.537Z","message":"Starting cluster with 1 processes"}1167vm-test-run-nixbot-gitlab> gitlab # [ 130.326278] gitlab-workhorse[1369]: time="2026-06-29T21:56:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1168vm-test-run-nixbot-gitlab> gitlab # [ 140.327598] gitlab-workhorse[1369]: time="2026-06-29T21:56:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1169vm-test-run-nixbot-gitlab> gitlab # [ 150.329173] gitlab-workhorse[1369]: time="2026-06-29T21:56:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1170vm-test-run-nixbot-gitlab> gitlab # [ 160.339484] gitlab-workhorse[1369]: time="2026-06-29T21:57:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1171vm-test-run-nixbot-gitlab> gitlab # [ 170.340619] gitlab-workhorse[1369]: time="2026-06-29T21:57:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1172vm-test-run-nixbot-gitlab> gitlab # [ 171.203756] bundle[2469]: W, [2026-06-29T21:57:19.117855 #2469] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1173vm-test-run-nixbot-gitlab> gitlab # [ 175.614831] sidekiq-cluster[2518]: 2026-06-29T21:57:23.529Z pid=2518 tid=tpa INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2518, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>445800448}1174vm-test-run-nixbot-gitlab> gitlab # [ 175.927953] sidekiq-cluster[2518]: W, [2026-06-29T21:57:23.843021 #2518] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1175vm-test-run-nixbot-gitlab> gitlab # [ 175.985070] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:23.900Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1176vm-test-run-nixbot-gitlab> gitlab # [ 179.183700] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.097Z","message":"GitLab reliable fetch activated!"}1177vm-test-run-nixbot-gitlab> gitlab # [ 179.184138] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.097Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1178vm-test-run-nixbot-gitlab> gitlab # [ 179.217317] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.132Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1179vm-test-run-nixbot-gitlab> gitlab # [ 179.224636] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.136Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1180vm-test-run-nixbot-gitlab> gitlab # [ 179.231520] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.139Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1181vm-test-run-nixbot-gitlab> gitlab # [ 179.239394] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.142Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1182vm-test-run-nixbot-gitlab> gitlab # [ 179.246519] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.146Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1183vm-test-run-nixbot-gitlab> gitlab # [ 179.253399] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.149Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1184vm-test-run-nixbot-gitlab> gitlab # [ 179.259166] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.151Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1185vm-test-run-nixbot-gitlab> gitlab # [ 179.264349] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.153Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1186vm-test-run-nixbot-gitlab> gitlab # [ 179.268935] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.155Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1187vm-test-run-nixbot-gitlab> gitlab # [ 179.273283] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.158Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1188vm-test-run-nixbot-gitlab> gitlab # [ 179.277792] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.160Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1189vm-test-run-nixbot-gitlab> gitlab # [ 179.281827] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.163Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1190vm-test-run-nixbot-gitlab> gitlab # [ 179.285376] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.165Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1191vm-test-run-nixbot-gitlab> gitlab # [ 179.289001] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.167Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1192vm-test-run-nixbot-gitlab> gitlab # [ 179.292347] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.169Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1193vm-test-run-nixbot-gitlab> gitlab # [ 179.295560] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.172Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1194vm-test-run-nixbot-gitlab> gitlab # [ 179.298891] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.174Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1195vm-test-run-nixbot-gitlab> gitlab # [ 179.301873] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.177Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1196vm-test-run-nixbot-gitlab> gitlab # [ 179.304627] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.180Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1197vm-test-run-nixbot-gitlab> gitlab # [ 179.307447] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.183Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1198vm-test-run-nixbot-gitlab> gitlab # [ 179.310262] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.186Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1199vm-test-run-nixbot-gitlab> gitlab # [ 179.313185] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.189Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1200vm-test-run-nixbot-gitlab> gitlab # [ 179.316119] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.191Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1201vm-test-run-nixbot-gitlab> gitlab # [ 179.318633] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.194Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1202vm-test-run-nixbot-gitlab> gitlab # [ 179.321251] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.196Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1203vm-test-run-nixbot-gitlab> gitlab # [ 179.323742] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.199Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1204vm-test-run-nixbot-gitlab> gitlab # [ 179.326226] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.202Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1205vm-test-run-nixbot-gitlab> gitlab # [ 179.328478] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.205Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1206vm-test-run-nixbot-gitlab> gitlab # [ 179.330761] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.208Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1207vm-test-run-nixbot-gitlab> gitlab # [ 179.333167] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.210Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1208vm-test-run-nixbot-gitlab> gitlab # [ 179.335484] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.213Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1209vm-test-run-nixbot-gitlab> gitlab # [ 179.337744] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.216Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1210vm-test-run-nixbot-gitlab> gitlab # [ 179.354978] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.268Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1211vm-test-run-nixbot-gitlab> gitlab # [ 179.358519] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.273Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1212vm-test-run-nixbot-gitlab> gitlab # [ 179.362284] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.276Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1213vm-test-run-nixbot-gitlab> gitlab # [ 179.365905] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.279Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1214vm-test-run-nixbot-gitlab> gitlab # [ 179.369443] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.282Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1215vm-test-run-nixbot-gitlab> gitlab # [ 179.372973] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.285Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1216vm-test-run-nixbot-gitlab> gitlab # [ 179.376358] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.289Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1217vm-test-run-nixbot-gitlab> gitlab # [ 179.379745] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.292Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1218vm-test-run-nixbot-gitlab> gitlab # [ 179.383249] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.294Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1219vm-test-run-nixbot-gitlab> gitlab # [ 179.386998] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.298Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1220vm-test-run-nixbot-gitlab> gitlab # [ 179.390629] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.300Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1221vm-test-run-nixbot-gitlab> gitlab # [ 179.394245] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.303Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1222vm-test-run-nixbot-gitlab> gitlab # [ 179.397540] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.305Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1223vm-test-run-nixbot-gitlab> gitlab # [ 179.400907] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.309Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1224vm-test-run-nixbot-gitlab> gitlab # [ 179.404196] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.311Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1225vm-test-run-nixbot-gitlab> gitlab # [ 179.407110] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.314Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1226vm-test-run-nixbot-gitlab> gitlab # [ 179.410041] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.316Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1227vm-test-run-nixbot-gitlab> gitlab # [ 179.412791] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.320Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1228vm-test-run-nixbot-gitlab> gitlab # [ 179.415424] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.322Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1229vm-test-run-nixbot-gitlab> gitlab # [ 179.418198] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.325Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1230vm-test-run-nixbot-gitlab> gitlab # [ 179.420785] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.328Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1231vm-test-run-nixbot-gitlab> gitlab # [ 179.428317] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.343Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1232vm-test-run-nixbot-gitlab> gitlab # [ 179.432213] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.347Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1233vm-test-run-nixbot-gitlab> gitlab # [ 179.435657] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.350Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1234vm-test-run-nixbot-gitlab> gitlab # [ 179.438638] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.353Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1235vm-test-run-nixbot-gitlab> gitlab # [ 179.441534] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.356Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1236vm-test-run-nixbot-gitlab> gitlab # [ 179.444513] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.359Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1237vm-test-run-nixbot-gitlab> gitlab # [ 179.447413] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.362Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1238vm-test-run-nixbot-gitlab> gitlab # [ 179.450518] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.364Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1239vm-test-run-nixbot-gitlab> gitlab # [ 179.453270] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.366Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1240vm-test-run-nixbot-gitlab> gitlab # [ 179.455906] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.369Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1241vm-test-run-nixbot-gitlab> gitlab # [ 179.458866] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.372Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1242vm-test-run-nixbot-gitlab> gitlab # [ 179.461797] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.377Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1243vm-test-run-nixbot-gitlab> gitlab # [ 179.464672] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.379Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1244vm-test-run-nixbot-gitlab> gitlab # [ 179.467355] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.382Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1245vm-test-run-nixbot-gitlab> gitlab # [ 179.470038] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.384Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [ 179.472745] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.386Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [ 179.475464] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.388Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [ 179.478199] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.390Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [ 179.480706] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.392Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [ 179.483074] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.394Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [ 179.485552] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.395Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [ 179.488063] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.398Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [ 179.490591] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.399Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [ 179.493260] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.402Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [ 179.495692] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.403Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [ 179.498542] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.406Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [ 179.500941] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.407Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [ 179.503207] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.410Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [ 179.505854] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.412Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [ 179.508459] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.413Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [ 179.510892] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.416Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [ 179.513358] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.418Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [ 179.515859] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.421Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [ 179.518413] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.423Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [ 179.521007] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.425Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [ 179.523405] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.427Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [ 179.525933] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.430Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [ 179.528161] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.432Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [ 179.530557] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.434Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [ 179.533071] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.436Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [ 179.535525] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.438Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [ 179.538280] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.440Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [ 179.540479] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.443Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [ 179.542986] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.446Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [ 179.545530] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.448Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [ 179.547946] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.450Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [ 179.551685] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.463Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [ 179.554170] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.466Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [ 179.556383] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.468Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [ 179.558623] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.471Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [ 179.560756] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.474Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [ 179.562966] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.477Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [ 179.565423] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.480Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [ 179.567778] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.482Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [ 179.570132] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.484Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [ 179.572451] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.487Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [ 179.574700] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:57:27.489Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [ 180.350858] gitlab-workhorse[1369]: time="2026-06-29T21:57:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1289vm-test-run-nixbot-gitlab> gitlab # [ 190.353873] gitlab-workhorse[1369]: time="2026-06-29T21:57:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1290vm-test-run-nixbot-gitlab> gitlab # [ 200.361969] gitlab-workhorse[1369]: time="2026-06-29T21:57:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1291vm-test-run-nixbot-gitlab> gitlab # [ 210.370365] gitlab-workhorse[1369]: time="2026-06-29T21:57:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1292vm-test-run-nixbot-gitlab> gitlab # [ 217.034021] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:04.947Z","message":"Booted Rails 7.2.3 application in production environment"}1293vm-test-run-nixbot-gitlab> gitlab # [ 217.034466] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:04.948Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1294vm-test-run-nixbot-gitlab> gitlab # [ 217.034651] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:04.948Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1295vm-test-run-nixbot-gitlab> gitlab # [ 217.034804] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:04.949Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1296vm-test-run-nixbot-gitlab> gitlab # [ 217.037177] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:04.952Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1297vm-test-run-nixbot-gitlab> gitlab # [ 219.690934] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:07.604Z","message":"Cleaning working queues"}1298vm-test-run-nixbot-gitlab> gitlab # [ 219.695482] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:07.608Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e1fb9c74563f2bab422b5398","created_at":"2026-06-29T21:56:26.057Z","trace_propagation_headers":{"sentry-trace":"c52f5a75cf7f4733946d72b305b8ef74-f8803023d0ed417e","baggage":"sentry-trace_id=c52f5a75cf7f4733946d72b305b8ef74,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"bef8b63188977f1bff7ae9e2027ca183","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:26.079Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e1fb9c74563f2bab422b5398: start","job_status":"start","queue_duration_s":101.528669,"scheduling_latency_s":101.528786}1299vm-test-run-nixbot-gitlab> gitlab # [ 220.163603] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:08.077Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e1fb9c74563f2bab422b5398","created_at":"2026-06-29T21:56:26.057Z","trace_propagation_headers":{"sentry-trace":"c52f5a75cf7f4733946d72b305b8ef74-f8803023d0ed417e","baggage":"sentry-trace_id=c52f5a75cf7f4733946d72b305b8ef74,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"bef8b63188977f1bff7ae9e2027ca183","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:26.079Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e1fb9c74563f2bab422b5398: done: 0.470204 sec","job_status":"done","queue_duration_s":101.528669,"scheduling_latency_s":101.528786,"redis_calls":29,"redis_duration_s":0.023978,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.015358,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006275,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20602692,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.470204,"completed_at":"2026-06-29T21:58:08.077Z","load_balancing_strategy":"primary","db_duration_s":0.127411,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1300vm-test-run-nixbot-gitlab> gitlab # [ 220.371556] gitlab-workhorse[1369]: time="2026-06-29T21:58:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1301vm-test-run-nixbot-gitlab> gitlab # [ 220.447615] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.361Z","pid":2469,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1302vm-test-run-nixbot-gitlab> gitlab # [ 220.453661] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.361Z","pid":2469,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1303vm-test-run-nixbot-gitlab> gitlab # [ 220.458853] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.361Z","pid":2469,"message":"! #\u003cThread:0x0000e2ce773ebc68 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1304vm-test-run-nixbot-gitlab> gitlab # [ 220.475127] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.361Z","pid":2469,"message":"! #\u003cThread:0x0000e2ce775ed4d0 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1305vm-test-run-nixbot-gitlab> gitlab # [ 220.491302] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.361Z","pid":2469,"message":"Use Ctrl-C to stop"}1306vm-test-run-nixbot-gitlab> gitlab # [ 220.495184] bundle[2469]: {"timestamp":"2026-06-29T21:58:08.362Z","pid":2469,"message":"Sending status to systemd every 1.0 sec"}1307vm-test-run-nixbot-gitlab> gitlab # [ 221.511493] bundle[2469]: {"timestamp":"2026-06-29T21:58:09.425Z","pid":2469,"message":"- Worker 0 (PID: 3371) booted in 1.01s, phase: 0"}1308vm-test-run-nixbot-gitlab> gitlab # [ 221.511958] bundle[2469]: {"timestamp":"2026-06-29T21:58:09.425Z","pid":2469,"message":"- Worker 1 (PID: 3375) booted in 0.99s, phase: 0"}1309vm-test-run-nixbot-gitlab> gitlab # [ 221.518769] systemd[1]: Started gitlab.service.1310vm-test-run-nixbot-gitlab> gitlab # [ 221.522406] systemd[1]: Reached target Common target for all GitLab services..1311vm-test-run-nixbot-gitlab> gitlab # [ 221.527410] systemd[1]: Starting setup-gitlab.service...1312vm-test-run-nixbot-gitlab> gitlab # [ 221.561915] setup-gitlab-start[3408]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1313vm-test-run-nixbot-gitlab> gitlab # [ 224.707391] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:58:12 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 31011314vm-test-run-nixbot-gitlab> gitlab # [ 224.715197] setup-gitlab-start[3431]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1315vm-test-run-nixbot-gitlab> gitlab # [ 224.717926] setup-gitlab-start[3432]: ++ jq -r .access_token1316vm-test-run-nixbot-gitlab> gitlab # [ 225.723996] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:58:13 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 9821317vm-test-run-nixbot-gitlab> gitlab # [ 225.729293] setup-gitlab-start[3408]: + BEARER=4c4aad8182ee4a044c738dfbdfa97f67cd027e23bbed093e57e463810a544e551318vm-test-run-nixbot-gitlab> gitlab # [ 225.731691] setup-gitlab-start[3408]: + AUTH='Authorization: Bearer 4c4aad8182ee4a044c738dfbdfa97f67cd027e23bbed093e57e463810a544e55'1319vm-test-run-nixbot-gitlab> gitlab # [ 225.741868] setup-gitlab-start[3408]: + curl -fs -X PUT -H 'Authorization: Bearer 4c4aad8182ee4a044c738dfbdfa97f67cd027e23bbed093e57e463810a544e55' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1320vm-test-run-nixbot-gitlab> gitlab # [ 226.750462] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:58:14 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 9931321vm-test-run-nixbot-gitlab> gitlab # [ 226.754647] setup-gitlab-start[3408]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1322vm-test-run-nixbot-gitlab> gitlab # [ 226.774591] runuser[3450]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1323vm-test-run-nixbot-gitlab> gitlab # [ 227.283520] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1324vm-test-run-nixbot-gitlab> gitlab # [ 227.290263] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1325vm-test-run-nixbot-gitlab> gitlab # [ 227.296660] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1326vm-test-run-nixbot-gitlab> gitlab # [ 228.036240] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1327vm-test-run-nixbot-gitlab> gitlab # [ 228.037134] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1328vm-test-run-nixbot-gitlab> gitlab # [ 228.049358] setup-gitlab-start[3451]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1329vm-test-run-nixbot-gitlab> gitlab # [ 232.998111] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:20.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"98402936eacfa9e8272a9dd5","created_at":"2026-06-29T21:58:20.853Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b258892ab649bb0e391615b753f2efb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:20.907Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-98402936eacfa9e8272a9dd5: start","job_status":"start","queue_duration_s":0.0034,"scheduling_latency_s":0.003462}1330vm-test-run-nixbot-gitlab> gitlab # [ 233.182237] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:21.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"98402936eacfa9e8272a9dd5","created_at":"2026-06-29T21:58:20.853Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b258892ab649bb0e391615b753f2efb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:20.907Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-98402936eacfa9e8272a9dd5: done: 0.185367 sec","job_status":"done","queue_duration_s":0.0034,"scheduling_latency_s":0.003462,"redis_calls":10,"redis_duration_s":0.049322,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004109,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.043328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00100216,"duration_s":0.185367,"completed_at":"2026-06-29T21:58:21.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331vm-test-run-nixbot-gitlab> gitlab # [ 233.271605] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:21.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ac03a151b7a8645df8798401","created_at":"2026-06-29T21:58:20.966Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e08dc677181b9fcc7574551694503b5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:20.984Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ac03a151b7a8645df8798401: start","job_status":"start","queue_duration_s":0.116646,"scheduling_latency_s":0.116693}1332vm-test-run-nixbot-gitlab> gitlab # [ 234.345611] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ac03a151b7a8645df8798401","created_at":"2026-06-29T21:58:20.966Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e08dc677181b9fcc7574551694503b5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:20.984Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ac03a151b7a8645df8798401: done: 1.158484 sec","job_status":"done","queue_duration_s":0.116646,"scheduling_latency_s":0.116693,"redis_calls":14,"redis_duration_s":0.004741,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001433,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.607887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00102466,"exclusive_lock_hold_duration_s":1.1154030799999999,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":1.158484,"completed_at":"2026-06-29T21:58:22.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.088722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333vm-test-run-nixbot-gitlab> gitlab # [ 234.346247] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"65cc6b56683e19f771c6f90a","created_at":"2026-06-29T21:58:21.046Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49435b3016f7c1be60359cdffcba8ba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.185Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-65cc6b56683e19f771c6f90a: start","job_status":"start","queue_duration_s":1.074981,"scheduling_latency_s":1.075004}1334vm-test-run-nixbot-gitlab> gitlab # [ 234.488760] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"65cc6b56683e19f771c6f90a","created_at":"2026-06-29T21:58:21.046Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49435b3016f7c1be60359cdffcba8ba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.185Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-65cc6b56683e19f771c6f90a: done: 0.142541 sec","job_status":"done","queue_duration_s":1.074981,"scheduling_latency_s":1.075004,"redis_calls":14,"redis_duration_s":0.011317,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000549,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005869,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004899,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.085,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048614,"exclusive_lock_hold_duration_s":0.10364062,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.142541,"completed_at":"2026-06-29T21:58:22.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.090177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335vm-test-run-nixbot-gitlab> gitlab # [ 234.602274] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c0c77470e3313b88211addf4","created_at":"2026-06-29T21:58:21.195Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ce1e1fe24876cc0f1ed65f1abf39b48","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.210Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c0c77470e3313b88211addf4: start","job_status":"start","queue_duration_s":1.19516,"scheduling_latency_s":1.195182}1336vm-test-run-nixbot-gitlab> gitlab # [ 234.617737] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c0c77470e3313b88211addf4","created_at":"2026-06-29T21:58:21.195Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ce1e1fe24876cc0f1ed65f1abf39b48","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.210Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c0c77470e3313b88211addf4: done: 0.121682 sec","job_status":"done","queue_duration_s":1.19516,"scheduling_latency_s":1.195182,"redis_calls":12,"redis_duration_s":0.0033940000000000003,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001159,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000622,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051854,"duration_s":0.121682,"completed_at":"2026-06-29T21:58:22.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337vm-test-run-nixbot-gitlab> gitlab # [ 234.661907] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0c7d412a94dbdee7a9c13a14","created_at":"2026-06-29T21:58:21.226Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42b72844d93303a1e52177a9959b0250","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:58:21.238Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0c7d412a94dbdee7a9c13a14: start","job_status":"start","queue_duration_s":1.294358,"scheduling_latency_s":1.294397}1338vm-test-run-nixbot-gitlab> gitlab # [ 234.677229] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.585Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1339vm-test-run-nixbot-gitlab> gitlab # [ 234.700394] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0c7d412a94dbdee7a9c13a14","created_at":"2026-06-29T21:58:21.226Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42b72844d93303a1e52177a9959b0250","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:58:21.238Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0c7d412a94dbdee7a9c13a14: done: 0.080653 sec","job_status":"done","queue_duration_s":1.294358,"scheduling_latency_s":1.294397,"redis_calls":10,"redis_duration_s":0.005545,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000939,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00111854,"duration_s":0.080653,"completed_at":"2026-06-29T21:58:22.613Z","load_balancing_strategy":"primary","db_duration_s":0.007085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340vm-test-run-nixbot-gitlab> gitlab # [ 234.791062] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1a8cc8eb0006dea338cf4273","created_at":"2026-06-29T21:58:21.277Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793386e7e3a54d5cda1a08da5f344502","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.286Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1a8cc8eb0006dea338cf4273: start","job_status":"start","queue_duration_s":1.419201,"scheduling_latency_s":1.419251}1341vm-test-run-nixbot-gitlab> gitlab # [ 234.929330] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.843Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1a8cc8eb0006dea338cf4273","created_at":"2026-06-29T21:58:21.277Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793386e7e3a54d5cda1a08da5f344502","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.286Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1a8cc8eb0006dea338cf4273: done: 0.137407 sec","job_status":"done","queue_duration_s":1.419201,"scheduling_latency_s":1.419251,"redis_calls":16,"redis_duration_s":0.005961999999999999,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003119,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00074,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063038,"duration_s":0.137407,"completed_at":"2026-06-29T21:58:22.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342vm-test-run-nixbot-gitlab> gitlab # [ 235.021197] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"dd8f065febec389ea010ba10","created_at":"2026-06-29T21:58:21.300Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa0cdee5a9f5b77fd577a8d1b238a2f4","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.310Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-dd8f065febec389ea010ba10: start","job_status":"start","queue_duration_s":1.545856,"scheduling_latency_s":1.545902}1343vm-test-run-nixbot-gitlab> gitlab # [ 235.078694] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"dd8f065febec389ea010ba10","created_at":"2026-06-29T21:58:21.300Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa0cdee5a9f5b77fd577a8d1b238a2f4","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.310Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-dd8f065febec389ea010ba10: done: 0.136417 sec","job_status":"done","queue_duration_s":1.545856,"scheduling_latency_s":1.545902,"redis_calls":10,"redis_duration_s":0.003699,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000656,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006484,"duration_s":0.136417,"completed_at":"2026-06-29T21:58:22.992Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344vm-test-run-nixbot-gitlab> gitlab # [ 235.170984] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:22.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b9e719b2b8580fe47e33b9b9","created_at":"2026-06-29T21:58:21.330Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.337Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b9e719b2b8580fe47e33b9b9: start","job_status":"start","queue_duration_s":1.662585,"scheduling_latency_s":1.662635}1345vm-test-run-nixbot-gitlab> gitlab # [ 235.287763] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b9e719b2b8580fe47e33b9b9","created_at":"2026-06-29T21:58:21.330Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.337Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b9e719b2b8580fe47e33b9b9: done: 0.202216 sec","job_status":"done","queue_duration_s":1.662585,"scheduling_latency_s":1.662635,"redis_calls":19,"redis_duration_s":0.031736,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002845,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003975,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.022264,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.104337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00373004,"duration_s":0.202216,"completed_at":"2026-06-29T21:58:23.201Z","load_balancing_strategy":"primary","db_duration_s":0.030294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346vm-test-run-nixbot-gitlab> gitlab # [ 235.388462] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b55acfb9690ad60c13027bba","created_at":"2026-06-29T21:58:21.344Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f838d6ff5a9ec918c94a7108bf057ad8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.350Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b55acfb9690ad60c13027bba: start","job_status":"start","queue_duration_s":1.951631,"scheduling_latency_s":1.951684}1347vm-test-run-nixbot-gitlab> gitlab # [ 235.535973] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b55acfb9690ad60c13027bba","created_at":"2026-06-29T21:58:21.344Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f838d6ff5a9ec918c94a7108bf057ad8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.350Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b55acfb9690ad60c13027bba: done: 0.14784 sec","job_status":"done","queue_duration_s":1.951631,"scheduling_latency_s":1.951684,"redis_calls":12,"redis_duration_s":0.019976,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003061,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007692,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00251136,"exclusive_lock_hold_duration_s":0.01790178,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.14784,"completed_at":"2026-06-29T21:58:23.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348vm-test-run-nixbot-gitlab> gitlab # [ 235.635644] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"121b4e20a92dc88c68318ab4","created_at":"2026-06-29T21:58:21.355Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfe84843b9abc1fc7fcd9662d9431f43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.362Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-121b4e20a92dc88c68318ab4: start","job_status":"start","queue_duration_s":2.09156,"scheduling_latency_s":2.091605}1349vm-test-run-nixbot-gitlab> gitlab # [ 235.657527] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.497Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1350vm-test-run-nixbot-gitlab> gitlab # [ 235.662613] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"121b4e20a92dc88c68318ab4","created_at":"2026-06-29T21:58:21.355Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfe84843b9abc1fc7fcd9662d9431f43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.362Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-121b4e20a92dc88c68318ab4: done: 0.083339 sec","job_status":"done","queue_duration_s":2.09156,"scheduling_latency_s":2.091605,"redis_calls":10,"redis_duration_s":0.020863,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006065,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00448522,"duration_s":0.083339,"completed_at":"2026-06-29T21:58:23.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351vm-test-run-nixbot-gitlab> gitlab # [ 235.708646] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"adfce524737d3fff13c8b730","created_at":"2026-06-29T21:58:21.379Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2fbce5745763afc616aacd3a7f3e1e3","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.385Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-adfce524737d3fff13c8b730: start","job_status":"start","queue_duration_s":2.163571,"scheduling_latency_s":2.163618}1352vm-test-run-nixbot-gitlab> gitlab # [ 235.774895] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"adfce524737d3fff13c8b730","created_at":"2026-06-29T21:58:21.379Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2fbce5745763afc616aacd3a7f3e1e3","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.385Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-adfce524737d3fff13c8b730: done: 0.140159 sec","job_status":"done","queue_duration_s":2.163571,"scheduling_latency_s":2.163618,"redis_calls":12,"redis_duration_s":0.02571,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.007174,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010387,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064448,"duration_s":0.140159,"completed_at":"2026-06-29T21:58:23.689Z","load_balancing_strategy":"primary","db_duration_s":0.004926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353vm-test-run-nixbot-gitlab> gitlab # [ 235.835269] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c01bd57c48f5ef8773064505","created_at":"2026-06-29T21:58:21.390Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa43aaf0782344e7ca81729f6f7bb31f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.398Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c01bd57c48f5ef8773064505: start","job_status":"start","queue_duration_s":2.298542,"scheduling_latency_s":2.298573}1354vm-test-run-nixbot-gitlab> gitlab # [ 235.853547] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.767Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c01bd57c48f5ef8773064505","created_at":"2026-06-29T21:58:21.390Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa43aaf0782344e7ca81729f6f7bb31f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.398Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c01bd57c48f5ef8773064505: done: 0.07057 sec","job_status":"done","queue_duration_s":2.298542,"scheduling_latency_s":2.298573,"redis_calls":11,"redis_duration_s":0.00324,"redis_read_bytes":137,"redis_write_bytes":1369,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000866,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053156,"duration_s":0.07057,"completed_at":"2026-06-29T21:58:23.767Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355vm-test-run-nixbot-gitlab> gitlab # [ 235.898947] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"23c72db144f458ec2ef22c4a","created_at":"2026-06-29T21:58:21.405Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eac5473b17fb386cf824eb56d93b9b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.411Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-23c72db144f458ec2ef22c4a: start","job_status":"start","queue_duration_s":2.360904,"scheduling_latency_s":2.360948}1356vm-test-run-nixbot-gitlab> gitlab # [ 235.957955] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"23c72db144f458ec2ef22c4a","created_at":"2026-06-29T21:58:21.405Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eac5473b17fb386cf824eb56d93b9b3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.411Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-23c72db144f458ec2ef22c4a: done: 0.100715 sec","job_status":"done","queue_duration_s":2.360904,"scheduling_latency_s":2.360948,"redis_calls":12,"redis_duration_s":0.014757,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000606,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007565,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00114416,"exclusive_lock_hold_duration_s":0.02222988,"duration_s":0.100715,"completed_at":"2026-06-29T21:58:23.872Z","load_balancing_strategy":"primary","db_duration_s":0.021154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357vm-test-run-nixbot-gitlab> gitlab # [ 236.058521] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:23.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2eaa28680d9f2e6b9799660e","created_at":"2026-06-29T21:58:21.416Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f816cecd567dd1e85fab322efeb59dc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.423Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2eaa28680d9f2e6b9799660e: start","job_status":"start","queue_duration_s":2.549601,"scheduling_latency_s":2.549648}1358vm-test-run-nixbot-gitlab> gitlab # [ 236.260505] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2eaa28680d9f2e6b9799660e","created_at":"2026-06-29T21:58:21.416Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f816cecd567dd1e85fab322efeb59dc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.423Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2eaa28680d9f2e6b9799660e: done: 0.201545 sec","job_status":"done","queue_duration_s":2.549601,"scheduling_latency_s":2.549648,"redis_calls":12,"redis_duration_s":0.025192,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00473,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00147952,"exclusive_lock_hold_duration_s":0.05225848,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.201545,"completed_at":"2026-06-29T21:58:24.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359vm-test-run-nixbot-gitlab> gitlab # [ 236.345250] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a7df70276e108b53d9ea188c","created_at":"2026-06-29T21:58:21.427Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8bbfdff03da0ee24c48558628ef26e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:58:21.433Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a7df70276e108b53d9ea188c: start","job_status":"start","queue_duration_s":2.759703,"scheduling_latency_s":2.759759}1360vm-test-run-nixbot-gitlab> gitlab # [ 236.421416] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a7df70276e108b53d9ea188c","created_at":"2026-06-29T21:58:21.427Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8bbfdff03da0ee24c48558628ef26e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:58:21.433Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a7df70276e108b53d9ea188c: done: 0.141336 sec","job_status":"done","queue_duration_s":2.759703,"scheduling_latency_s":2.759759,"redis_calls":10,"redis_duration_s":0.028088,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004137,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01679,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007161,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00414636,"duration_s":0.141336,"completed_at":"2026-06-29T21:58:24.334Z","load_balancing_strategy":"primary","db_duration_s":0.01654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361vm-test-run-nixbot-gitlab> gitlab # [ 236.507811] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71f08af87ad32564c020bc00","created_at":"2026-06-29T21:58:21.441Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54d01fa90f9c447febe92452e41ab5a0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.449Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71f08af87ad32564c020bc00: start","job_status":"start","queue_duration_s":2.898371,"scheduling_latency_s":2.898433}1362vm-test-run-nixbot-gitlab> gitlab # [ 236.615488] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71f08af87ad32564c020bc00","created_at":"2026-06-29T21:58:21.441Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54d01fa90f9c447febe92452e41ab5a0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.449Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71f08af87ad32564c020bc00: done: 0.181869 sec","job_status":"done","queue_duration_s":2.898371,"scheduling_latency_s":2.898433,"redis_calls":13,"redis_duration_s":0.036723,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004154,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.020079,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.01249,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03866554,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.181869,"completed_at":"2026-06-29T21:58:24.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04502,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1363vm-test-run-nixbot-gitlab> gitlab # [ 236.706554] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"614e894a552b953ef23fc9c6","created_at":"2026-06-29T21:58:21.455Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ef5e650978b62c354021dace2cf3880","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.475Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-614e894a552b953ef23fc9c6: start","job_status":"start","queue_duration_s":3.058889,"scheduling_latency_s":3.058939}1364vm-test-run-nixbot-gitlab> gitlab # [ 236.823209] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"614e894a552b953ef23fc9c6","created_at":"2026-06-29T21:58:21.455Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ef5e650978b62c354021dace2cf3880","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.475Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-614e894a552b953ef23fc9c6: done: 0.203201 sec","job_status":"done","queue_duration_s":3.058889,"scheduling_latency_s":3.058939,"redis_calls":10,"redis_duration_s":0.003441,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000671,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.091,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.091,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037486,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.203201,"completed_at":"2026-06-29T21:58:24.737Z","load_balancing_strategy":"primary","db_duration_s":0.102496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365vm-test-run-nixbot-gitlab> gitlab # [ 236.914894] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8841826c65aa39eb708fcdb5","created_at":"2026-06-29T21:58:21.543Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d063abf1f9111940406eb7fd0561455d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:58:21.561Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8841826c65aa39eb708fcdb5: start","job_status":"start","queue_duration_s":3.181965,"scheduling_latency_s":3.182016}1366vm-test-run-nixbot-gitlab> gitlab # [ 236.931636] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.830Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1367vm-test-run-nixbot-gitlab> gitlab # [ 236.978135] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8841826c65aa39eb708fcdb5","created_at":"2026-06-29T21:58:21.543Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d063abf1f9111940406eb7fd0561455d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:58:21.561Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8841826c65aa39eb708fcdb5: done: 0.148513 sec","job_status":"done","queue_duration_s":3.181965,"scheduling_latency_s":3.182016,"redis_calls":10,"redis_duration_s":0.041189,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003583,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.024799,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00318234,"duration_s":0.148513,"completed_at":"2026-06-29T21:58:24.891Z","load_balancing_strategy":"primary","db_duration_s":0.00389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368vm-test-run-nixbot-gitlab> gitlab # [ 237.077063] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:24.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a1c69a82857dc03e8ede810e","created_at":"2026-06-29T21:58:21.577Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e32af32c2de7920131d594314bbfcef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.595Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a1c69a82857dc03e8ede810e: start","job_status":"start","queue_duration_s":3.395746,"scheduling_latency_s":3.395792}1369vm-test-run-nixbot-gitlab> gitlab # [ 237.168781] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.083Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1370vm-test-run-nixbot-gitlab> gitlab # [ 237.194914] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a1c69a82857dc03e8ede810e","created_at":"2026-06-29T21:58:21.577Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e32af32c2de7920131d594314bbfcef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.595Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a1c69a82857dc03e8ede810e: done: 0.116929 sec","job_status":"done","queue_duration_s":3.395746,"scheduling_latency_s":3.395792,"redis_calls":10,"redis_duration_s":0.009454,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002637,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00070728,"duration_s":0.116929,"completed_at":"2026-06-29T21:58:25.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371vm-test-run-nixbot-gitlab> gitlab # [ 237.290429] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.114Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7f4c0e424e1ec6106c7e1bad","created_at":"2026-06-29T21:58:21.600Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.617Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7f4c0e424e1ec6106c7e1bad: start","job_status":"start","queue_duration_s":3.497096,"scheduling_latency_s":3.497141}1372vm-test-run-nixbot-gitlab> gitlab # [ 237.811256] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7f4c0e424e1ec6106c7e1bad","created_at":"2026-06-29T21:58:21.600Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.617Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7f4c0e424e1ec6106c7e1bad: done: 0.609981 sec","job_status":"done","queue_duration_s":3.497096,"scheduling_latency_s":3.497141,"redis_calls":71,"redis_duration_s":0.154935,"redis_read_bytes":120,"redis_write_bytes":42574,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.010753,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00332,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.136602,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.274155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00387292,"duration_s":0.609981,"completed_at":"2026-06-29T21:58:25.724Z","load_balancing_strategy":"primary","db_duration_s":0.023647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373vm-test-run-nixbot-gitlab> gitlab # [ 237.918767] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9308fb8609b07a4f6a57d06a","created_at":"2026-06-29T21:58:21.636Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6ec0099395f200910d7954e4b0a7392","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.649Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9308fb8609b07a4f6a57d06a: start","job_status":"start","queue_duration_s":4.087358,"scheduling_latency_s":4.08741}1374vm-test-run-nixbot-gitlab> gitlab # [ 238.010942] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:25.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9308fb8609b07a4f6a57d06a","created_at":"2026-06-29T21:58:21.636Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6ec0099395f200910d7954e4b0a7392","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.649Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9308fb8609b07a4f6a57d06a: done: 0.187706 sec","job_status":"done","queue_duration_s":4.087358,"scheduling_latency_s":4.08741,"redis_calls":12,"redis_duration_s":0.102417,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.074818,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010429,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00304484,"exclusive_lock_hold_duration_s":0.01707864,"duration_s":0.187706,"completed_at":"2026-06-29T21:58:25.924Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375vm-test-run-nixbot-gitlab> gitlab # [ 238.401032] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:26.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5c2ae0a5f39e1929b5dfc0c7","created_at":"2026-06-29T21:58:21.680Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da949b82c567d26c748e6cc0a915808d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.689Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5c2ae0a5f39e1929b5dfc0c7: start","job_status":"start","queue_duration_s":4.625958,"scheduling_latency_s":4.625985}1376vm-test-run-nixbot-gitlab> gitlab # [ 238.466350] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:26.381Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1377vm-test-run-nixbot-gitlab> gitlab # [ 238.508921] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:26.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5c2ae0a5f39e1929b5dfc0c7","created_at":"2026-06-29T21:58:21.680Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da949b82c567d26c748e6cc0a915808d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.689Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5c2ae0a5f39e1929b5dfc0c7: done: 0.107407 sec","job_status":"done","queue_duration_s":4.625958,"scheduling_latency_s":4.625985,"redis_calls":10,"redis_duration_s":0.034134000000000005,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.01416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002235,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00275602,"duration_s":0.107407,"completed_at":"2026-06-29T21:58:26.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378vm-test-run-nixbot-gitlab> gitlab # [ 238.596789] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:26.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2ec0c32ed00a2dfc3cfa1a77","created_at":"2026-06-29T21:58:21.707Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1747cab21cb16fa75efe0cbc4fbca5d9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.717Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2ec0c32ed00a2dfc3cfa1a77: start","job_status":"start","queue_duration_s":4.719116,"scheduling_latency_s":4.719167}1379vm-test-run-nixbot-gitlab> gitlab # [ 239.913098] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2ec0c32ed00a2dfc3cfa1a77","created_at":"2026-06-29T21:58:21.707Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1747cab21cb16fa75efe0cbc4fbca5d9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.717Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2ec0c32ed00a2dfc3cfa1a77: done: 1.390323 sec","job_status":"done","queue_duration_s":4.719116,"scheduling_latency_s":4.719167,"redis_calls":574,"redis_duration_s":0.775739,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00662,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.761189,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.720765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003254,"duration_s":1.390323,"completed_at":"2026-06-29T21:58:27.826Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380vm-test-run-nixbot-gitlab> gitlab # [ 239.913589] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e08af9bc926531c5329219bb","created_at":"2026-06-29T21:58:21.728Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a25be0af4f7b5319e03e7bb62116124d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.741Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e08af9bc926531c5329219bb: start","job_status":"start","queue_duration_s":6.086993,"scheduling_latency_s":6.087019}1381vm-test-run-nixbot-gitlab> gitlab # [ 239.999643] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.913Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1382vm-test-run-nixbot-gitlab> gitlab # [ 240.053911] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e08af9bc926531c5329219bb","created_at":"2026-06-29T21:58:21.728Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a25be0af4f7b5319e03e7bb62116124d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:58:21.741Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e08af9bc926531c5329219bb: done: 0.103435 sec","job_status":"done","queue_duration_s":6.086993,"scheduling_latency_s":6.087019,"redis_calls":10,"redis_duration_s":0.003767,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000838,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068684,"duration_s":0.103435,"completed_at":"2026-06-29T21:58:27.931Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.070315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383vm-test-run-nixbot-gitlab> gitlab # [ 240.126998] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.934Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"678e98ce1c358191b4659c16","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-678e98ce1c358191b4659c16: start","job_status":"start","queue_duration_s":4.774081,"scheduling_latency_s":4.774118}1384vm-test-run-nixbot-gitlab> gitlab # [ 240.146589] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.962Z","message":"No existing merge request to be cleaned up."}1385vm-test-run-nixbot-gitlab> gitlab # [ 240.149009] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.985Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"678e98ce1c358191b4659c16","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-678e98ce1c358191b4659c16: done: 0.051349 sec","job_status":"done","queue_duration_s":4.774081,"scheduling_latency_s":4.774118,"redis_calls":13,"redis_duration_s":0.003735,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047254,"duration_s":0.051349,"completed_at":"2026-06-29T21:58:27.985Z","load_balancing_strategy":"primary","db_duration_s":0.006487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386vm-test-run-nixbot-gitlab> gitlab # [ 240.196389] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:27.987Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a8895a4fad49c7a5e1366c2","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a8895a4fad49c7a5e1366c2: start","job_status":"start","queue_duration_s":4.827251,"scheduling_latency_s":4.827284}1387vm-test-run-nixbot-gitlab> gitlab # [ 240.209666] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.001Z","message":"No existing merge request to be cleaned up."}1388vm-test-run-nixbot-gitlab> gitlab # [ 240.211561] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.010Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a8895a4fad49c7a5e1366c2","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a8895a4fad49c7a5e1366c2: done: 0.023433 sec","job_status":"done","queue_duration_s":4.827251,"scheduling_latency_s":4.827284,"redis_calls":10,"redis_duration_s":0.0019229999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023433,"completed_at":"2026-06-29T21:58:28.010Z","load_balancing_strategy":"primary","db_duration_s":0.001851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389vm-test-run-nixbot-gitlab> gitlab # [ 240.252657] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.012Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7deb17a51a3bb8f36593fe7d","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7deb17a51a3bb8f36593fe7d: start","job_status":"start","queue_duration_s":4.85227,"scheduling_latency_s":4.852294}1390vm-test-run-nixbot-gitlab> gitlab # [ 240.265097] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.024Z","message":"No existing merge request to be cleaned up."}1391vm-test-run-nixbot-gitlab> gitlab # [ 240.266989] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.032Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7deb17a51a3bb8f36593fe7d","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7deb17a51a3bb8f36593fe7d: done: 0.019997 sec","job_status":"done","queue_duration_s":4.85227,"scheduling_latency_s":4.852294,"redis_calls":10,"redis_duration_s":0.001572,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019997,"completed_at":"2026-06-29T21:58:28.032Z","load_balancing_strategy":"primary","db_duration_s":0.001522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392vm-test-run-nixbot-gitlab> gitlab # [ 240.307134] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.034Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5768142304080d463d767cef","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5768142304080d463d767cef: start","job_status":"start","queue_duration_s":4.87384,"scheduling_latency_s":4.873862}1393vm-test-run-nixbot-gitlab> gitlab # [ 240.319404] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.044Z","message":"No existing merge request to be cleaned up."}1394vm-test-run-nixbot-gitlab> gitlab # [ 240.321204] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.052Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5768142304080d463d767cef","created_at":"2026-06-29T21:58:23.048Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-bbc1054d5f944133","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7aff316fcc833b567d9fc8b44a6ccd58","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:23.160Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5768142304080d463d767cef: done: 0.018619 sec","job_status":"done","queue_duration_s":4.87384,"scheduling_latency_s":4.873862,"redis_calls":10,"redis_duration_s":0.001753,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018619,"completed_at":"2026-06-29T21:58:28.052Z","load_balancing_strategy":"primary","db_duration_s":0.001682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395vm-test-run-nixbot-gitlab> gitlab # [ 240.360698] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84cbfd206d6fe99b361b7f24","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84cbfd206d6fe99b361b7f24: start","job_status":"start","queue_duration_s":2.376797,"scheduling_latency_s":2.376817}1396vm-test-run-nixbot-gitlab> gitlab # [ 240.373797] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84cbfd206d6fe99b361b7f24","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84cbfd206d6fe99b361b7f24: done: 0.020507 sec","job_status":"done","queue_duration_s":2.376797,"scheduling_latency_s":2.376817,"redis_calls":16,"redis_duration_s":0.002221,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026588,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020507,"completed_at":"2026-06-29T21:58:28.074Z","load_balancing_strategy":"primary","db_duration_s":0.00032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397vm-test-run-nixbot-gitlab> gitlab # [ 240.415839] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3287895b712b310797577c6e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3287895b712b310797577c6e: start","job_status":"start","queue_duration_s":2.398599,"scheduling_latency_s":2.398619}1398vm-test-run-nixbot-gitlab> gitlab # [ 240.429217] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3287895b712b310797577c6e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3287895b712b310797577c6e: done: 0.015371 sec","job_status":"done","queue_duration_s":2.398599,"scheduling_latency_s":2.398619,"redis_calls":13,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015371,"completed_at":"2026-06-29T21:58:28.091Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399vm-test-run-nixbot-gitlab> gitlab # [ 240.469825] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e511c8cc4a56514640a343c7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e511c8cc4a56514640a343c7: start","job_status":"start","queue_duration_s":2.41508,"scheduling_latency_s":2.415098}1400vm-test-run-nixbot-gitlab> gitlab # [ 240.483145] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e511c8cc4a56514640a343c7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e511c8cc4a56514640a343c7: done: 0.014187 sec","job_status":"done","queue_duration_s":2.41508,"scheduling_latency_s":2.415098,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014187,"completed_at":"2026-06-29T21:58:28.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401vm-test-run-nixbot-gitlab> gitlab # [ 240.524210] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"091c2b1df294ee92fac461b9","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-091c2b1df294ee92fac461b9: start","job_status":"start","queue_duration_s":2.430361,"scheduling_latency_s":2.430379}1402vm-test-run-nixbot-gitlab> gitlab # [ 240.536609] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"091c2b1df294ee92fac461b9","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-091c2b1df294ee92fac461b9: done: 0.014663 sec","job_status":"done","queue_duration_s":2.430361,"scheduling_latency_s":2.430379,"redis_calls":13,"redis_duration_s":0.0017629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014663,"completed_at":"2026-06-29T21:58:28.122Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403vm-test-run-nixbot-gitlab> gitlab # [ 240.576899] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f64ee97cccd908e60a7ccb7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f64ee97cccd908e60a7ccb7: start","job_status":"start","queue_duration_s":2.446211,"scheduling_latency_s":2.446229}1404vm-test-run-nixbot-gitlab> gitlab # [ 240.589734] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f64ee97cccd908e60a7ccb7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f64ee97cccd908e60a7ccb7: done: 0.01486 sec","job_status":"done","queue_duration_s":2.446211,"scheduling_latency_s":2.446229,"redis_calls":13,"redis_duration_s":0.001886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01486,"completed_at":"2026-06-29T21:58:28.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405vm-test-run-nixbot-gitlab> gitlab # [ 240.630525] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdf5f3a7ffd463145ef53048","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdf5f3a7ffd463145ef53048: start","job_status":"start","queue_duration_s":2.462115,"scheduling_latency_s":2.462131}1406vm-test-run-nixbot-gitlab> gitlab # [ 240.643477] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdf5f3a7ffd463145ef53048","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdf5f3a7ffd463145ef53048: done: 0.015314 sec","job_status":"done","queue_duration_s":2.462115,"scheduling_latency_s":2.462131,"redis_calls":13,"redis_duration_s":0.0016250000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-06-29T21:58:28.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407vm-test-run-nixbot-gitlab> gitlab # [ 240.684691] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9a4e84831705e673cf99301","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9a4e84831705e673cf99301: start","job_status":"start","queue_duration_s":2.478656,"scheduling_latency_s":2.478674}1408vm-test-run-nixbot-gitlab> gitlab # [ 240.697528] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9a4e84831705e673cf99301","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9a4e84831705e673cf99301: done: 0.014262 sec","job_status":"done","queue_duration_s":2.478656,"scheduling_latency_s":2.478674,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014262,"completed_at":"2026-06-29T21:58:28.170Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409vm-test-run-nixbot-gitlab> gitlab # [ 240.736995] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"103a4ce0873522a1ffb2f268","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-103a4ce0873522a1ffb2f268: start","job_status":"start","queue_duration_s":2.494287,"scheduling_latency_s":2.494306}1410vm-test-run-nixbot-gitlab> gitlab # [ 240.749604] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"103a4ce0873522a1ffb2f268","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-103a4ce0873522a1ffb2f268: done: 0.014595 sec","job_status":"done","queue_duration_s":2.494287,"scheduling_latency_s":2.494306,"redis_calls":13,"redis_duration_s":0.0015400000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014595,"completed_at":"2026-06-29T21:58:28.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411vm-test-run-nixbot-gitlab> gitlab # [ 240.791680] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c27b117353ad8dc2ed3c65","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c27b117353ad8dc2ed3c65: start","job_status":"start","queue_duration_s":2.509708,"scheduling_latency_s":2.509726}1412vm-test-run-nixbot-gitlab> gitlab # [ 240.804931] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c27b117353ad8dc2ed3c65","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c27b117353ad8dc2ed3c65: done: 0.014804 sec","job_status":"done","queue_duration_s":2.509708,"scheduling_latency_s":2.509726,"redis_calls":13,"redis_duration_s":0.001764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014804,"completed_at":"2026-06-29T21:58:28.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413vm-test-run-nixbot-gitlab> gitlab # [ 240.846806] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad78b313a18f83b4e07f8fd1","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad78b313a18f83b4e07f8fd1: start","job_status":"start","queue_duration_s":2.525659,"scheduling_latency_s":2.525678}1414vm-test-run-nixbot-gitlab> gitlab # [ 240.859831] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad78b313a18f83b4e07f8fd1","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad78b313a18f83b4e07f8fd1: done: 0.014772 sec","job_status":"done","queue_duration_s":2.525659,"scheduling_latency_s":2.525678,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014772,"completed_at":"2026-06-29T21:58:28.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415vm-test-run-nixbot-gitlab> gitlab # [ 240.899574] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"711c0a692722d39e963e49f7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-711c0a692722d39e963e49f7: start","job_status":"start","queue_duration_s":2.541497,"scheduling_latency_s":2.541513}1416vm-test-run-nixbot-gitlab> gitlab # [ 240.912765] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"711c0a692722d39e963e49f7","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-711c0a692722d39e963e49f7: done: 0.014297 sec","job_status":"done","queue_duration_s":2.541497,"scheduling_latency_s":2.541513,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014297,"completed_at":"2026-06-29T21:58:28.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417vm-test-run-nixbot-gitlab> gitlab # [ 240.952324] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e720282855ba98928cc46d4d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e720282855ba98928cc46d4d: start","job_status":"start","queue_duration_s":2.556932,"scheduling_latency_s":2.55695}1418vm-test-run-nixbot-gitlab> gitlab # [ 240.966205] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e720282855ba98928cc46d4d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e720282855ba98928cc46d4d: done: 0.015248 sec","job_status":"done","queue_duration_s":2.556932,"scheduling_latency_s":2.55695,"redis_calls":13,"redis_duration_s":0.00195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015248,"completed_at":"2026-06-29T21:58:28.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419vm-test-run-nixbot-gitlab> gitlab # [ 241.005683] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e0ce0411b449ac650e3ffb","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e0ce0411b449ac650e3ffb: start","job_status":"start","queue_duration_s":2.573264,"scheduling_latency_s":2.57328}1420vm-test-run-nixbot-gitlab> gitlab # [ 241.018307] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e0ce0411b449ac650e3ffb","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e0ce0411b449ac650e3ffb: done: 0.014129 sec","job_status":"done","queue_duration_s":2.573264,"scheduling_latency_s":2.57328,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014129,"completed_at":"2026-06-29T21:58:28.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [ 241.058636] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdc4712cf7c6eb03ba4a0a2c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdc4712cf7c6eb03ba4a0a2c: start","job_status":"start","queue_duration_s":2.588442,"scheduling_latency_s":2.588458}1422vm-test-run-nixbot-gitlab> gitlab # [ 241.070998] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdc4712cf7c6eb03ba4a0a2c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdc4712cf7c6eb03ba4a0a2c: done: 0.014797 sec","job_status":"done","queue_duration_s":2.588442,"scheduling_latency_s":2.588458,"redis_calls":13,"redis_duration_s":0.001632,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014797,"completed_at":"2026-06-29T21:58:28.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [ 241.111622] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85c48ce539cc091b504ae35d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85c48ce539cc091b504ae35d: start","job_status":"start","queue_duration_s":2.605383,"scheduling_latency_s":2.605403}1424vm-test-run-nixbot-gitlab> gitlab # [ 241.125111] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85c48ce539cc091b504ae35d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85c48ce539cc091b504ae35d: done: 0.016027 sec","job_status":"done","queue_duration_s":2.605383,"scheduling_latency_s":2.605403,"redis_calls":13,"redis_duration_s":0.001771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016027,"completed_at":"2026-06-29T21:58:28.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425vm-test-run-nixbot-gitlab> gitlab # [ 241.164627] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c9342479a8115d709dca509","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c9342479a8115d709dca509: start","job_status":"start","queue_duration_s":2.622491,"scheduling_latency_s":2.62251}1426vm-test-run-nixbot-gitlab> gitlab # [ 241.178808] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c9342479a8115d709dca509","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c9342479a8115d709dca509: done: 0.014921 sec","job_status":"done","queue_duration_s":2.622491,"scheduling_latency_s":2.62251,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014921,"completed_at":"2026-06-29T21:58:28.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427vm-test-run-nixbot-gitlab> gitlab # [ 241.219449] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"449119df1d17062d76b45e90","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-449119df1d17062d76b45e90: start","job_status":"start","queue_duration_s":2.638635,"scheduling_latency_s":2.638652}1428vm-test-run-nixbot-gitlab> gitlab # [ 241.232853] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"449119df1d17062d76b45e90","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-449119df1d17062d76b45e90: done: 0.016467 sec","job_status":"done","queue_duration_s":2.638635,"scheduling_latency_s":2.638652,"redis_calls":13,"redis_duration_s":0.0015970000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016467,"completed_at":"2026-06-29T21:58:28.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429vm-test-run-nixbot-gitlab> gitlab # [ 241.274369] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7504d2972839ea2f021c9378","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7504d2972839ea2f021c9378: start","job_status":"start","queue_duration_s":2.656188,"scheduling_latency_s":2.656206}1430vm-test-run-nixbot-gitlab> gitlab # [ 241.287822] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7504d2972839ea2f021c9378","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7504d2972839ea2f021c9378: done: 0.015883 sec","job_status":"done","queue_duration_s":2.656188,"scheduling_latency_s":2.656206,"redis_calls":13,"redis_duration_s":0.001816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015883,"completed_at":"2026-06-29T21:58:28.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431vm-test-run-nixbot-gitlab> gitlab # [ 241.328323] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e3eb7e3c3b2d8ff4ddfc3e0","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e3eb7e3c3b2d8ff4ddfc3e0: start","job_status":"start","queue_duration_s":2.67323,"scheduling_latency_s":2.673249}1432vm-test-run-nixbot-gitlab> gitlab # [ 241.341424] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e3eb7e3c3b2d8ff4ddfc3e0","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e3eb7e3c3b2d8ff4ddfc3e0: done: 0.01639 sec","job_status":"done","queue_duration_s":2.67323,"scheduling_latency_s":2.673249,"redis_calls":13,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01639,"completed_at":"2026-06-29T21:58:28.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433vm-test-run-nixbot-gitlab> gitlab # [ 241.382573] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea7a022ea983c5bf0e02085","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea7a022ea983c5bf0e02085: start","job_status":"start","queue_duration_s":2.690832,"scheduling_latency_s":2.690849}1434vm-test-run-nixbot-gitlab> gitlab # [ 241.397296] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea7a022ea983c5bf0e02085","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea7a022ea983c5bf0e02085: done: 0.015299 sec","job_status":"done","queue_duration_s":2.690832,"scheduling_latency_s":2.690849,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015299,"completed_at":"2026-06-29T21:58:28.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [ 241.438601] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3555c9e8971ddcc82e05c424","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3555c9e8971ddcc82e05c424: start","job_status":"start","queue_duration_s":2.707238,"scheduling_latency_s":2.707256}1436vm-test-run-nixbot-gitlab> gitlab # [ 241.451897] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3555c9e8971ddcc82e05c424","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3555c9e8971ddcc82e05c424: done: 0.015974 sec","job_status":"done","queue_duration_s":2.707238,"scheduling_latency_s":2.707256,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015974,"completed_at":"2026-06-29T21:58:28.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [ 241.492593] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d1ae4312f7649fcc378ee8","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d1ae4312f7649fcc378ee8: start","job_status":"start","queue_duration_s":2.725543,"scheduling_latency_s":2.725563}1438vm-test-run-nixbot-gitlab> gitlab # [ 241.505642] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d1ae4312f7649fcc378ee8","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d1ae4312f7649fcc378ee8: done: 0.0198 sec","job_status":"done","queue_duration_s":2.725543,"scheduling_latency_s":2.725563,"redis_calls":13,"redis_duration_s":0.0017429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0198,"completed_at":"2026-06-29T21:58:28.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439vm-test-run-nixbot-gitlab> gitlab # [ 241.546041] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78819bccdfdd9fab35aa648","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78819bccdfdd9fab35aa648: start","job_status":"start","queue_duration_s":2.746512,"scheduling_latency_s":2.746528}1440vm-test-run-nixbot-gitlab> gitlab # [ 241.559284] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78819bccdfdd9fab35aa648","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78819bccdfdd9fab35aa648: done: 0.014347 sec","job_status":"done","queue_duration_s":2.746512,"scheduling_latency_s":2.746528,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014347,"completed_at":"2026-06-29T21:58:28.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441vm-test-run-nixbot-gitlab> gitlab # [ 241.601764] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e20bb5e2441138a77ef25d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e20bb5e2441138a77ef25d: start","job_status":"start","queue_duration_s":2.761921,"scheduling_latency_s":2.761938}1442vm-test-run-nixbot-gitlab> gitlab # [ 241.614996] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e20bb5e2441138a77ef25d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e20bb5e2441138a77ef25d: done: 0.014503 sec","job_status":"done","queue_duration_s":2.761921,"scheduling_latency_s":2.761938,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014503,"completed_at":"2026-06-29T21:58:28.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [ 241.655883] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7254ac66f0bf779778b834","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7254ac66f0bf779778b834: start","job_status":"start","queue_duration_s":2.777966,"scheduling_latency_s":2.777984}1444vm-test-run-nixbot-gitlab> gitlab # [ 241.669217] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7254ac66f0bf779778b834","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7254ac66f0bf779778b834: done: 0.01463 sec","job_status":"done","queue_duration_s":2.777966,"scheduling_latency_s":2.777984,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01463,"completed_at":"2026-06-29T21:58:28.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445vm-test-run-nixbot-gitlab> gitlab # [ 241.709348] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96068cb95c839c24e9f75ba4","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96068cb95c839c24e9f75ba4: start","job_status":"start","queue_duration_s":2.793643,"scheduling_latency_s":2.79366}1446vm-test-run-nixbot-gitlab> gitlab # [ 241.722042] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96068cb95c839c24e9f75ba4","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96068cb95c839c24e9f75ba4: done: 0.014099 sec","job_status":"done","queue_duration_s":2.793643,"scheduling_latency_s":2.79366,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014099,"completed_at":"2026-06-29T21:58:28.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447vm-test-run-nixbot-gitlab> gitlab # [ 241.764039] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3956996fdabc1eddb105a5","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3956996fdabc1eddb105a5: start","job_status":"start","queue_duration_s":2.808772,"scheduling_latency_s":2.808789}1448vm-test-run-nixbot-gitlab> gitlab # [ 241.776949] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3956996fdabc1eddb105a5","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3956996fdabc1eddb105a5: done: 0.014263 sec","job_status":"done","queue_duration_s":2.808772,"scheduling_latency_s":2.808789,"redis_calls":13,"redis_duration_s":0.0016070000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014263,"completed_at":"2026-06-29T21:58:28.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [ 241.819657] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad28b5437353958aaba06606","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad28b5437353958aaba06606: start","job_status":"start","queue_duration_s":2.824154,"scheduling_latency_s":2.82417}1450vm-test-run-nixbot-gitlab> gitlab # [ 241.833271] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad28b5437353958aaba06606","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad28b5437353958aaba06606: done: 0.159571 sec","job_status":"done","queue_duration_s":2.824154,"scheduling_latency_s":2.82417,"redis_calls":13,"redis_duration_s":0.002041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.159571,"completed_at":"2026-06-29T21:58:28.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [ 241.873023] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e82831d62523f8ef41b99771","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e82831d62523f8ef41b99771: start","job_status":"start","queue_duration_s":2.987658,"scheduling_latency_s":2.987702}1452vm-test-run-nixbot-gitlab> gitlab # [ 241.886163] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e82831d62523f8ef41b99771","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e82831d62523f8ef41b99771: done: 0.039537 sec","job_status":"done","queue_duration_s":2.987658,"scheduling_latency_s":2.987702,"redis_calls":13,"redis_duration_s":0.005496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039537,"completed_at":"2026-06-29T21:58:28.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [ 241.927278] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b66035e160b80b6e241a55e4","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b66035e160b80b6e241a55e4: start","job_status":"start","queue_duration_s":3.02981,"scheduling_latency_s":3.029849}1454vm-test-run-nixbot-gitlab> gitlab # [ 241.940918] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b66035e160b80b6e241a55e4","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b66035e160b80b6e241a55e4: done: 0.027415 sec","job_status":"done","queue_duration_s":3.02981,"scheduling_latency_s":3.029849,"redis_calls":13,"redis_duration_s":0.0044399999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002492,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027415,"completed_at":"2026-06-29T21:58:28.734Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [ 241.980734] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d00c953e77894713e3e161f","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d00c953e77894713e3e161f: start","job_status":"start","queue_duration_s":3.05908,"scheduling_latency_s":3.059103}1456vm-test-run-nixbot-gitlab> gitlab # [ 241.994013] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d00c953e77894713e3e161f","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d00c953e77894713e3e161f: done: 0.022102 sec","job_status":"done","queue_duration_s":3.05908,"scheduling_latency_s":3.059103,"redis_calls":13,"redis_duration_s":0.003901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002307,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022102,"completed_at":"2026-06-29T21:58:28.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [ 242.034060] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34436decec8f45946c275e01","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34436decec8f45946c275e01: start","job_status":"start","queue_duration_s":3.083596,"scheduling_latency_s":3.083619}1458vm-test-run-nixbot-gitlab> gitlab # [ 242.046732] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34436decec8f45946c275e01","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34436decec8f45946c275e01: done: 0.020782 sec","job_status":"done","queue_duration_s":3.083596,"scheduling_latency_s":3.083619,"redis_calls":13,"redis_duration_s":0.003866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002282,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020782,"completed_at":"2026-06-29T21:58:28.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [ 242.086974] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a4b7febfd74b380597bf38","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a4b7febfd74b380597bf38: start","job_status":"start","queue_duration_s":3.106075,"scheduling_latency_s":3.106096}1460vm-test-run-nixbot-gitlab> gitlab # [ 242.100155] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a4b7febfd74b380597bf38","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a4b7febfd74b380597bf38: done: 0.018438 sec","job_status":"done","queue_duration_s":3.106075,"scheduling_latency_s":3.106096,"redis_calls":13,"redis_duration_s":0.003053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018438,"completed_at":"2026-06-29T21:58:28.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [ 242.139388] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2384408bb5f32bbe5eb6ceed","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2384408bb5f32bbe5eb6ceed: start","job_status":"start","queue_duration_s":3.125983,"scheduling_latency_s":3.126}1462vm-test-run-nixbot-gitlab> gitlab # [ 242.152536] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2384408bb5f32bbe5eb6ceed","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2384408bb5f32bbe5eb6ceed: done: 0.016259 sec","job_status":"done","queue_duration_s":3.125983,"scheduling_latency_s":3.126,"redis_calls":13,"redis_duration_s":0.0032300000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016259,"completed_at":"2026-06-29T21:58:28.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [ 242.192161] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d8fd8fd99009738e4081f9a","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d8fd8fd99009738e4081f9a: start","job_status":"start","queue_duration_s":3.143669,"scheduling_latency_s":3.143686}1464vm-test-run-nixbot-gitlab> gitlab # [ 242.206252] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d8fd8fd99009738e4081f9a","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d8fd8fd99009738e4081f9a: done: 0.014821 sec","job_status":"done","queue_duration_s":3.143669,"scheduling_latency_s":3.143686,"redis_calls":13,"redis_duration_s":0.002657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001723,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014821,"completed_at":"2026-06-29T21:58:28.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465vm-test-run-nixbot-gitlab> gitlab # [ 242.246480] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22023793fd17ac61e9b74802","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22023793fd17ac61e9b74802: start","job_status":"start","queue_duration_s":3.160176,"scheduling_latency_s":3.160193}1466vm-test-run-nixbot-gitlab> gitlab # [ 242.259489] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22023793fd17ac61e9b74802","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22023793fd17ac61e9b74802: done: 0.016301 sec","job_status":"done","queue_duration_s":3.160176,"scheduling_latency_s":3.160193,"redis_calls":13,"redis_duration_s":0.003423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002387,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016301,"completed_at":"2026-06-29T21:58:28.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467vm-test-run-nixbot-gitlab> gitlab # [ 242.298722] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fcbeca36d500c258b0b14cd","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fcbeca36d500c258b0b14cd: start","job_status":"start","queue_duration_s":3.178149,"scheduling_latency_s":3.178166}1468vm-test-run-nixbot-gitlab> gitlab # [ 242.311779] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fcbeca36d500c258b0b14cd","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fcbeca36d500c258b0b14cd: done: 0.015444 sec","job_status":"done","queue_duration_s":3.178149,"scheduling_latency_s":3.178166,"redis_calls":13,"redis_duration_s":0.003013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00106,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015444,"completed_at":"2026-06-29T21:58:28.870Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469vm-test-run-nixbot-gitlab> gitlab # [ 242.351947] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6400021cccc77f18ac37a863","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6400021cccc77f18ac37a863: start","job_status":"start","queue_duration_s":3.195155,"scheduling_latency_s":3.195172}1470vm-test-run-nixbot-gitlab> gitlab # [ 242.364705] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6400021cccc77f18ac37a863","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6400021cccc77f18ac37a863: done: 0.015593 sec","job_status":"done","queue_duration_s":3.195155,"scheduling_latency_s":3.195172,"redis_calls":13,"redis_duration_s":0.00289,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015593,"completed_at":"2026-06-29T21:58:28.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471vm-test-run-nixbot-gitlab> gitlab # [ 242.405632] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8daa9d64d0d7d8c05183517c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8daa9d64d0d7d8c05183517c: start","job_status":"start","queue_duration_s":3.21212,"scheduling_latency_s":3.212136}1472vm-test-run-nixbot-gitlab> gitlab # [ 242.418780] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8daa9d64d0d7d8c05183517c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8daa9d64d0d7d8c05183517c: done: 0.015531 sec","job_status":"done","queue_duration_s":3.21212,"scheduling_latency_s":3.212136,"redis_calls":13,"redis_duration_s":0.003056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015531,"completed_at":"2026-06-29T21:58:28.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [ 242.460163] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a70a772a986d1a56c5fb213","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a70a772a986d1a56c5fb213: start","job_status":"start","queue_duration_s":3.229118,"scheduling_latency_s":3.229134}1474vm-test-run-nixbot-gitlab> gitlab # [ 242.473063] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a70a772a986d1a56c5fb213","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a70a772a986d1a56c5fb213: done: 0.029962 sec","job_status":"done","queue_duration_s":3.229118,"scheduling_latency_s":3.229134,"redis_calls":13,"redis_duration_s":0.003221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029962,"completed_at":"2026-06-29T21:58:28.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475vm-test-run-nixbot-gitlab> gitlab # [ 242.513744] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9672e4e4845a1c334a4381","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9672e4e4845a1c334a4381: start","job_status":"start","queue_duration_s":3.260552,"scheduling_latency_s":3.260569}1476vm-test-run-nixbot-gitlab> gitlab # [ 242.526737] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9672e4e4845a1c334a4381","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9672e4e4845a1c334a4381: done: 0.016145 sec","job_status":"done","queue_duration_s":3.260552,"scheduling_latency_s":3.260569,"redis_calls":13,"redis_duration_s":0.0037070000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002634,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016145,"completed_at":"2026-06-29T21:58:28.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477vm-test-run-nixbot-gitlab> gitlab # [ 242.566172] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c9495c36c3213fd925a8e4e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c9495c36c3213fd925a8e4e: start","job_status":"start","queue_duration_s":3.278419,"scheduling_latency_s":3.278437}1478vm-test-run-nixbot-gitlab> gitlab # [ 242.579224] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c9495c36c3213fd925a8e4e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c9495c36c3213fd925a8e4e: done: 0.015197 sec","job_status":"done","queue_duration_s":3.278419,"scheduling_latency_s":3.278437,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015197,"completed_at":"2026-06-29T21:58:28.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479vm-test-run-nixbot-gitlab> gitlab # [ 242.620947] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d371c21edec23f1ed0899a5e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d371c21edec23f1ed0899a5e: start","job_status":"start","queue_duration_s":3.294896,"scheduling_latency_s":3.294913}1480vm-test-run-nixbot-gitlab> gitlab # [ 242.633443] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d371c21edec23f1ed0899a5e","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d371c21edec23f1ed0899a5e: done: 0.014524 sec","job_status":"done","queue_duration_s":3.294896,"scheduling_latency_s":3.294913,"redis_calls":13,"redis_duration_s":0.002768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014524,"completed_at":"2026-06-29T21:58:28.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481vm-test-run-nixbot-gitlab> gitlab # [ 242.674223] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:28.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaf30666977f3037500cc552","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaf30666977f3037500cc552: start","job_status":"start","queue_duration_s":3.310732,"scheduling_latency_s":3.31075}1482vm-test-run-nixbot-gitlab> gitlab # [ 242.687683] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaf30666977f3037500cc552","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaf30666977f3037500cc552: done: 0.015431 sec","job_status":"done","queue_duration_s":3.310732,"scheduling_latency_s":3.31075,"redis_calls":13,"redis_duration_s":0.00301,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015431,"completed_at":"2026-06-29T21:58:29.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483vm-test-run-nixbot-gitlab> gitlab # [ 242.727972] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40cf349f7e80a40699aed536","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40cf349f7e80a40699aed536: start","job_status":"start","queue_duration_s":3.32777,"scheduling_latency_s":3.327789}1484vm-test-run-nixbot-gitlab> gitlab # [ 242.741082] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40cf349f7e80a40699aed536","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40cf349f7e80a40699aed536: done: 0.014781 sec","job_status":"done","queue_duration_s":3.32777,"scheduling_latency_s":3.327789,"redis_calls":13,"redis_duration_s":0.001921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014781,"completed_at":"2026-06-29T21:58:29.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485vm-test-run-nixbot-gitlab> gitlab # [ 242.780529] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82296bb8d548755e7e51b1d2","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82296bb8d548755e7e51b1d2: start","job_status":"start","queue_duration_s":3.343685,"scheduling_latency_s":3.343701}1486vm-test-run-nixbot-gitlab> gitlab # [ 242.793419] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82296bb8d548755e7e51b1d2","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82296bb8d548755e7e51b1d2: done: 0.014735 sec","job_status":"done","queue_duration_s":3.343685,"scheduling_latency_s":3.343701,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014735,"completed_at":"2026-06-29T21:58:29.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [ 242.833335] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88b4af86bfde7196427550c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88b4af86bfde7196427550c: start","job_status":"start","queue_duration_s":3.359606,"scheduling_latency_s":3.359636}1488vm-test-run-nixbot-gitlab> gitlab # [ 242.846325] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88b4af86bfde7196427550c","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88b4af86bfde7196427550c: done: 0.015574 sec","job_status":"done","queue_duration_s":3.359606,"scheduling_latency_s":3.359636,"redis_calls":13,"redis_duration_s":0.002065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001403,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015574,"completed_at":"2026-06-29T21:58:29.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [ 242.885801] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97c61a5c7cc22b55f2da9c65","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97c61a5c7cc22b55f2da9c65: start","job_status":"start","queue_duration_s":3.376221,"scheduling_latency_s":3.376237}1490vm-test-run-nixbot-gitlab> gitlab # [ 242.898204] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97c61a5c7cc22b55f2da9c65","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97c61a5c7cc22b55f2da9c65: done: 0.013966 sec","job_status":"done","queue_duration_s":3.376221,"scheduling_latency_s":3.376237,"redis_calls":13,"redis_duration_s":0.001655,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013966,"completed_at":"2026-06-29T21:58:29.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [ 242.939378] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05ac0aadea0dc30710f1d73d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05ac0aadea0dc30710f1d73d: start","job_status":"start","queue_duration_s":3.391309,"scheduling_latency_s":3.391326}1492vm-test-run-nixbot-gitlab> gitlab # [ 242.952622] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05ac0aadea0dc30710f1d73d","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05ac0aadea0dc30710f1d73d: done: 0.014259 sec","job_status":"done","queue_duration_s":3.391309,"scheduling_latency_s":3.391326,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014259,"completed_at":"2026-06-29T21:58:29.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [ 242.991867] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22802d66b165183680bddd6a","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22802d66b165183680bddd6a: start","job_status":"start","queue_duration_s":3.406797,"scheduling_latency_s":3.406817}1494vm-test-run-nixbot-gitlab> gitlab # [ 243.004822] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:58:29.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22802d66b165183680bddd6a","created_at":"2026-06-29T21:58:25.194Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-e7d8175ad28f40a0","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4e43d31fe46dc36479f2e859b95f14cf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:58:25.677Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22802d66b165183680bddd6a: done: 0.014554 sec","job_status":"done","queue_duration_s":3.406797,"scheduling_latency_s":3.406817,"redis_calls":13,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014554,"completed_at":"2026-06-29T21:58:29.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [ 269.569189] setup-gitlab-start[3451]: W, [2026-06-29T21:58:57.483363 #3451] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1496vm-test-run-nixbot-gitlab> gitlab # [ 275.796570] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:03.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"be51b04b2ae5919fb3f520eb","created_at":"2026-06-29T21:59:03.689Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e17f163ea76f7d5b68ab6c51d986ed80","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.706Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-be51b04b2ae5919fb3f520eb: start","job_status":"start","queue_duration_s":0.003576,"scheduling_latency_s":0.003639}1497vm-test-run-nixbot-gitlab> gitlab # [ 275.898044] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:03.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"be51b04b2ae5919fb3f520eb","created_at":"2026-06-29T21:59:03.689Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e17f163ea76f7d5b68ab6c51d986ed80","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.706Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-be51b04b2ae5919fb3f520eb: done: 0.102562 sec","job_status":"done","queue_duration_s":0.003576,"scheduling_latency_s":0.003639,"redis_calls":8,"redis_duration_s":0.0040230000000000005,"redis_read_bytes":133,"redis_write_bytes":745,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000778,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.102562,"completed_at":"2026-06-29T21:59:03.812Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [ 275.995786] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:03.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2845e140962d9fa0c669662f","created_at":"2026-06-29T21:59:03.746Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47aa2fd40ee7ca82c72185f58f039a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.758Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2845e140962d9fa0c669662f: start","job_status":"start","queue_duration_s":0.060031,"scheduling_latency_s":0.060087}1499vm-test-run-nixbot-gitlab> gitlab # [ 276.774852] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2845e140962d9fa0c669662f","created_at":"2026-06-29T21:59:03.746Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47aa2fd40ee7ca82c72185f58f039a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.758Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2845e140962d9fa0c669662f: done: 0.870508 sec","job_status":"done","queue_duration_s":0.060031,"scheduling_latency_s":0.060087,"redis_calls":9,"redis_duration_s":0.003582,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.202,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.202,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.379799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00153614,"exclusive_lock_hold_duration_s":0.82366174,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.870508,"completed_at":"2026-06-29T21:59:04.689Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.499656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [ 276.777532] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c7542a4889f44867e0b42ad7","created_at":"2026-06-29T21:59:03.807Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d181fb4fef21b0cce6ace08d327ca2ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.815Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c7542a4889f44867e0b42ad7: start","job_status":"start","queue_duration_s":0.876233,"scheduling_latency_s":0.876263}1501vm-test-run-nixbot-gitlab> gitlab # [ 276.939153] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c7542a4889f44867e0b42ad7","created_at":"2026-06-29T21:59:03.807Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d181fb4fef21b0cce6ace08d327ca2ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.815Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c7542a4889f44867e0b42ad7: done: 0.160426 sec","job_status":"done","queue_duration_s":0.876233,"scheduling_latency_s":0.876263,"redis_calls":9,"redis_duration_s":0.003843,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00126482,"exclusive_lock_hold_duration_s":0.0346769,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.160426,"completed_at":"2026-06-29T21:59:04.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [ 277.021795] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.855Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f965518954abd252287ee41a","created_at":"2026-06-29T21:59:03.828Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86feba5dabbaeda54895839d004f8921","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.832Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f965518954abd252287ee41a: start","job_status":"start","queue_duration_s":1.022566,"scheduling_latency_s":1.022605}1503vm-test-run-nixbot-gitlab> gitlab # [ 277.040361] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f965518954abd252287ee41a","created_at":"2026-06-29T21:59:03.828Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86feba5dabbaeda54895839d004f8921","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.832Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f965518954abd252287ee41a: done: 0.025521 sec","job_status":"done","queue_duration_s":1.022566,"scheduling_latency_s":1.022605,"redis_calls":7,"redis_duration_s":0.002342,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000681,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025521,"completed_at":"2026-06-29T21:59:04.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [ 277.087100] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"211586376282711c9e19551a","created_at":"2026-06-29T21:59:03.868Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"701fa0179b43bc23e8c6036f5ebbe668","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:59:03.874Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-211586376282711c9e19551a: start","job_status":"start","queue_duration_s":1.007644,"scheduling_latency_s":1.00767}1505vm-test-run-nixbot-gitlab> gitlab # [ 277.102696] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.894Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1506vm-test-run-nixbot-gitlab> gitlab # [ 277.106137] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"211586376282711c9e19551a","created_at":"2026-06-29T21:59:03.868Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"701fa0179b43bc23e8c6036f5ebbe668","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:59:03.874Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-211586376282711c9e19551a: done: 0.01924 sec","job_status":"done","queue_duration_s":1.007644,"scheduling_latency_s":1.00767,"redis_calls":7,"redis_duration_s":0.002101,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01924,"completed_at":"2026-06-29T21:59:04.901Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507vm-test-run-nixbot-gitlab> gitlab # [ 277.145594] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.903Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a83da4faccfa679708e63580","created_at":"2026-06-29T21:59:03.927Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"680d0329f054d6d4da55e605ba042742","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.931Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a83da4faccfa679708e63580: start","job_status":"start","queue_duration_s":0.972396,"scheduling_latency_s":0.972417}1508vm-test-run-nixbot-gitlab> gitlab # [ 277.159586] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.926Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a83da4faccfa679708e63580","created_at":"2026-06-29T21:59:03.927Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"680d0329f054d6d4da55e605ba042742","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.931Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a83da4faccfa679708e63580: done: 0.023096 sec","job_status":"done","queue_duration_s":0.972396,"scheduling_latency_s":0.972417,"redis_calls":7,"redis_duration_s":0.0019030000000000002,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023096,"completed_at":"2026-06-29T21:59:04.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003789,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509vm-test-run-nixbot-gitlab> gitlab # [ 277.200175] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0e001052b95bc14c6a674e1b","created_at":"2026-06-29T21:59:03.964Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.967Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0e001052b95bc14c6a674e1b: start","job_status":"start","queue_duration_s":0.961343,"scheduling_latency_s":0.961363}1510vm-test-run-nixbot-gitlab> gitlab # [ 277.214261] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0e001052b95bc14c6a674e1b","created_at":"2026-06-29T21:59:03.964Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.967Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0e001052b95bc14c6a674e1b: done: 0.026026 sec","job_status":"done","queue_duration_s":0.961343,"scheduling_latency_s":0.961363,"redis_calls":14,"redis_duration_s":0.0035399999999999997,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.00047,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002312,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026026,"completed_at":"2026-06-29T21:59:04.954Z","load_balancing_strategy":"primary","db_duration_s":0.000455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [ 277.257258] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ca09c65185ef1d2b410c672a","created_at":"2026-06-29T21:59:03.980Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e8177251e9511d04de31ae42bd2c8a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.983Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ca09c65185ef1d2b410c672a: start","job_status":"start","queue_duration_s":0.977198,"scheduling_latency_s":0.977218}1512vm-test-run-nixbot-gitlab> gitlab # [ 277.271694] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ca09c65185ef1d2b410c672a","created_at":"2026-06-29T21:59:03.980Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e8177251e9511d04de31ae42bd2c8a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.983Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ca09c65185ef1d2b410c672a: done: 0.017553 sec","job_status":"done","queue_duration_s":0.977198,"scheduling_latency_s":0.977218,"redis_calls":9,"redis_duration_s":0.002948,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048064,"exclusive_lock_hold_duration_s":0.00182138,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017553,"completed_at":"2026-06-29T21:59:04.978Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [ 277.315692] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1c6e752f10a599252147ed5c","created_at":"2026-06-29T21:59:03.993Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30cf577507f9f75241ea4fcce28688ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.997Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1c6e752f10a599252147ed5c: start","job_status":"start","queue_duration_s":0.982217,"scheduling_latency_s":0.982234}1514vm-test-run-nixbot-gitlab> gitlab # [ 277.331052] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.986Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1515vm-test-run-nixbot-gitlab> gitlab # [ 277.335156] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1c6e752f10a599252147ed5c","created_at":"2026-06-29T21:59:03.993Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30cf577507f9f75241ea4fcce28688ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:03.997Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1c6e752f10a599252147ed5c: done: 0.014466 sec","job_status":"done","queue_duration_s":0.982217,"scheduling_latency_s":0.982234,"redis_calls":7,"redis_duration_s":0.002378,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014466,"completed_at":"2026-06-29T21:59:04.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [ 277.377334] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:04.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"318c1f59874297581a5d1702","created_at":"2026-06-29T21:59:04.028Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9689207b2fefecd0207266f468345128","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.032Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-318c1f59874297581a5d1702: start","job_status":"start","queue_duration_s":0.964588,"scheduling_latency_s":0.964607}1517vm-test-run-nixbot-gitlab> gitlab # [ 277.393402] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"318c1f59874297581a5d1702","created_at":"2026-06-29T21:59:04.028Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9689207b2fefecd0207266f468345128","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.032Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-318c1f59874297581a5d1702: done: 0.014705 sec","job_status":"done","queue_duration_s":0.964588,"scheduling_latency_s":0.964607,"redis_calls":7,"redis_duration_s":0.001906,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014705,"completed_at":"2026-06-29T21:59:05.012Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518vm-test-run-nixbot-gitlab> gitlab # [ 277.436599] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8209b8055363414b16cb61b2","created_at":"2026-06-29T21:59:04.059Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569d3e81af005737f32830ba4b19451f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.064Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8209b8055363414b16cb61b2: start","job_status":"start","queue_duration_s":0.948533,"scheduling_latency_s":0.948551}1519vm-test-run-nixbot-gitlab> gitlab # [ 277.450892] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8209b8055363414b16cb61b2","created_at":"2026-06-29T21:59:04.059Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569d3e81af005737f32830ba4b19451f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.064Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8209b8055363414b16cb61b2: done: 0.015743 sec","job_status":"done","queue_duration_s":0.948533,"scheduling_latency_s":0.948551,"redis_calls":9,"redis_duration_s":0.0013579999999999998,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004408,"exclusive_lock_hold_duration_s":0.00212452,"duration_s":0.015743,"completed_at":"2026-06-29T21:59:05.029Z","load_balancing_strategy":"primary","db_duration_s":0.000433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520vm-test-run-nixbot-gitlab> gitlab # [ 277.491503] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9087bd9e5ff9d8863615bf3f","created_at":"2026-06-29T21:59:04.085Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9ad7046e5cc1ab566ee34328333f988","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.095Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9087bd9e5ff9d8863615bf3f: start","job_status":"start","queue_duration_s":0.934379,"scheduling_latency_s":0.934395}1521vm-test-run-nixbot-gitlab> gitlab # [ 277.507576] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.061Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9087bd9e5ff9d8863615bf3f","created_at":"2026-06-29T21:59:04.085Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9ad7046e5cc1ab566ee34328333f988","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.095Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9087bd9e5ff9d8863615bf3f: done: 0.031179 sec","job_status":"done","queue_duration_s":0.934379,"scheduling_latency_s":0.934395,"redis_calls":9,"redis_duration_s":0.001376,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000688,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058242,"exclusive_lock_hold_duration_s":0.01198436,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.031179,"completed_at":"2026-06-29T21:59:05.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [ 277.555450] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"49f87131d328a79605f106f1","created_at":"2026-06-29T21:59:04.111Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a93d6402a9fb0ddc036892a00163e20b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:59:04.123Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-49f87131d328a79605f106f1: start","job_status":"start","queue_duration_s":0.939272,"scheduling_latency_s":0.939289}1523vm-test-run-nixbot-gitlab> gitlab # [ 277.570030] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"49f87131d328a79605f106f1","created_at":"2026-06-29T21:59:04.111Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a93d6402a9fb0ddc036892a00163e20b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:59:04.123Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-49f87131d328a79605f106f1: done: 0.018797 sec","job_status":"done","queue_duration_s":0.939272,"scheduling_latency_s":0.939289,"redis_calls":7,"redis_duration_s":0.000974,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018797,"completed_at":"2026-06-29T21:59:05.081Z","load_balancing_strategy":"primary","db_duration_s":0.000611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524vm-test-run-nixbot-gitlab> gitlab # [ 277.611824] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e7342405622d43452d827b60","created_at":"2026-06-29T21:59:04.152Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91c46fcc9ddee38d0eeb8180c1ba7f24","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.161Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e7342405622d43452d827b60: start","job_status":"start","queue_duration_s":0.921032,"scheduling_latency_s":0.92105}1525vm-test-run-nixbot-gitlab> gitlab # [ 277.627554] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e7342405622d43452d827b60","created_at":"2026-06-29T21:59:04.152Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91c46fcc9ddee38d0eeb8180c1ba7f24","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.161Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e7342405622d43452d827b60: done: 0.018948 sec","job_status":"done","queue_duration_s":0.921032,"scheduling_latency_s":0.92105,"redis_calls":10,"redis_duration_s":0.001469,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00326864,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.018948,"completed_at":"2026-06-29T21:59:05.101Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001174,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1526vm-test-run-nixbot-gitlab> gitlab # [ 277.672271] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"54557cc9a70be31c3ed04f38","created_at":"2026-06-29T21:59:04.187Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b95947939bf63bb259322f7ed135e1ec","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.198Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-54557cc9a70be31c3ed04f38: start","job_status":"start","queue_duration_s":0.903821,"scheduling_latency_s":0.903837}1527vm-test-run-nixbot-gitlab> gitlab # [ 277.686664] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"54557cc9a70be31c3ed04f38","created_at":"2026-06-29T21:59:04.187Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b95947939bf63bb259322f7ed135e1ec","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.198Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-54557cc9a70be31c3ed04f38: done: 0.014766 sec","job_status":"done","queue_duration_s":0.903821,"scheduling_latency_s":0.903837,"redis_calls":7,"redis_duration_s":0.000961,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.014766,"completed_at":"2026-06-29T21:59:05.117Z","load_balancing_strategy":"primary","db_duration_s":0.001204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [ 277.729607] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7cfa8072fce8b922ea21933d","created_at":"2026-06-29T21:59:04.269Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51645fd813ff801bf2ff1091ea8d6713","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:59:04.272Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7cfa8072fce8b922ea21933d: start","job_status":"start","queue_duration_s":0.845994,"scheduling_latency_s":0.846013}1529vm-test-run-nixbot-gitlab> gitlab # [ 277.743479] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.125Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1530vm-test-run-nixbot-gitlab> gitlab # [ 277.747454] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7cfa8072fce8b922ea21933d","created_at":"2026-06-29T21:59:04.269Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51645fd813ff801bf2ff1091ea8d6713","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:59:04.272Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7cfa8072fce8b922ea21933d: done: 0.013095 sec","job_status":"done","queue_duration_s":0.845994,"scheduling_latency_s":0.846013,"redis_calls":7,"redis_duration_s":0.001139,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013095,"completed_at":"2026-06-29T21:59:05.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [ 277.787630] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7e450f17edad33f0e897733c","created_at":"2026-06-29T21:59:04.291Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74d303d5fd4860f24ba75fe43a2c4cec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.296Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7e450f17edad33f0e897733c: start","job_status":"start","queue_duration_s":0.83608,"scheduling_latency_s":0.836097}1532vm-test-run-nixbot-gitlab> gitlab # [ 277.803098] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.140Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1533vm-test-run-nixbot-gitlab> gitlab # [ 277.806635] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7e450f17edad33f0e897733c","created_at":"2026-06-29T21:59:04.291Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74d303d5fd4860f24ba75fe43a2c4cec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.296Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7e450f17edad33f0e897733c: done: 0.012592 sec","job_status":"done","queue_duration_s":0.83608,"scheduling_latency_s":0.836097,"redis_calls":7,"redis_duration_s":0.000984,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012592,"completed_at":"2026-06-29T21:59:05.144Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [ 277.848432] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"739a46cff1b529640e9cd5c1","created_at":"2026-06-29T21:59:04.312Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.316Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-739a46cff1b529640e9cd5c1: start","job_status":"start","queue_duration_s":0.829297,"scheduling_latency_s":0.829313}1535vm-test-run-nixbot-gitlab> gitlab # [ 277.889652] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"739a46cff1b529640e9cd5c1","created_at":"2026-06-29T21:59:04.312Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.316Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-739a46cff1b529640e9cd5c1: done: 0.097957 sec","job_status":"done","queue_duration_s":0.829297,"scheduling_latency_s":0.829313,"redis_calls":60,"redis_duration_s":0.015518,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.002216,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.00787,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.097957,"completed_at":"2026-06-29T21:59:05.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [ 277.960310] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9f645f002a8e0968a73e217f","created_at":"2026-06-29T21:59:04.362Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f14da219a1a01ade9b4eb60d81b5900","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.365Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9f645f002a8e0968a73e217f: start","job_status":"start","queue_duration_s":0.879355,"scheduling_latency_s":0.879372}1537vm-test-run-nixbot-gitlab> gitlab # [ 277.980469] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.253Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1538vm-test-run-nixbot-gitlab> gitlab # [ 277.985577] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9f645f002a8e0968a73e217f","created_at":"2026-06-29T21:59:04.362Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f14da219a1a01ade9b4eb60d81b5900","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.365Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9f645f002a8e0968a73e217f: done: 0.012835 sec","job_status":"done","queue_duration_s":0.879355,"scheduling_latency_s":0.879372,"redis_calls":7,"redis_duration_s":0.0009989999999999999,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012835,"completed_at":"2026-06-29T21:59:05.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539vm-test-run-nixbot-gitlab> gitlab # [ 278.030471] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"03d477d5956ebe77ba6f53da","created_at":"2026-06-29T21:59:04.390Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e95430ec3492e13a15aee4c716dacbd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.395Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-03d477d5956ebe77ba6f53da: start","job_status":"start","queue_duration_s":0.863857,"scheduling_latency_s":0.863874}1540vm-test-run-nixbot-gitlab> gitlab # [ 278.044679] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.578Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"03d477d5956ebe77ba6f53da","created_at":"2026-06-29T21:59:04.390Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e95430ec3492e13a15aee4c716dacbd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.395Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-03d477d5956ebe77ba6f53da: done: 0.319642 sec","job_status":"done","queue_duration_s":0.863857,"scheduling_latency_s":0.863874,"redis_calls":571,"redis_duration_s":0.066488,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.065726,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.292869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.319642,"completed_at":"2026-06-29T21:59:05.578Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541vm-test-run-nixbot-gitlab> gitlab # [ 278.085616] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2252bf668be36cc8900b5436","created_at":"2026-06-29T21:59:04.414Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ea176e280a9eb4adac95843e031687","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.426Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2252bf668be36cc8900b5436: start","job_status":"start","queue_duration_s":1.153394,"scheduling_latency_s":1.153416}1542vm-test-run-nixbot-gitlab> gitlab # [ 278.100203] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.588Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1543vm-test-run-nixbot-gitlab> gitlab # [ 278.103490] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2252bf668be36cc8900b5436","created_at":"2026-06-29T21:59:04.414Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-972674e0124e4575","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0ea176e280a9eb4adac95843e031687","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.426Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2252bf668be36cc8900b5436: done: 0.013578 sec","job_status":"done","queue_duration_s":1.153394,"scheduling_latency_s":1.153416,"redis_calls":7,"redis_duration_s":0.001041,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000311,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013578,"completed_at":"2026-06-29T21:59:05.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [ 278.145838] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.594Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"06f6d7f5322f21084fef67a8","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-06f6d7f5322f21084fef67a8: start","job_status":"start","queue_duration_s":0.645515,"scheduling_latency_s":0.645529}1545vm-test-run-nixbot-gitlab> gitlab # [ 278.158916] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.606Z","message":"No existing merge request to be cleaned up."}1546vm-test-run-nixbot-gitlab> gitlab # [ 278.160748] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.614Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"06f6d7f5322f21084fef67a8","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-06f6d7f5322f21084fef67a8: done: 0.019369 sec","job_status":"done","queue_duration_s":0.645515,"scheduling_latency_s":0.645529,"redis_calls":10,"redis_duration_s":0.001635,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019369,"completed_at":"2026-06-29T21:59:05.614Z","load_balancing_strategy":"primary","db_duration_s":0.001721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [ 278.200667] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.615Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"786ca84ed4ce3c792fc90ec7","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-786ca84ed4ce3c792fc90ec7: start","job_status":"start","queue_duration_s":0.666001,"scheduling_latency_s":0.666015}1548vm-test-run-nixbot-gitlab> gitlab # [ 278.213558] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.625Z","message":"No existing merge request to be cleaned up."}1549vm-test-run-nixbot-gitlab> gitlab # [ 278.215208] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.632Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"786ca84ed4ce3c792fc90ec7","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-786ca84ed4ce3c792fc90ec7: done: 0.017239 sec","job_status":"done","queue_duration_s":0.666001,"scheduling_latency_s":0.666015,"redis_calls":10,"redis_duration_s":0.001388,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000809,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017239,"completed_at":"2026-06-29T21:59:05.632Z","load_balancing_strategy":"primary","db_duration_s":0.001283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [ 278.255005] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.634Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe0881d4557fa7f3e69581a4","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe0881d4557fa7f3e69581a4: start","job_status":"start","queue_duration_s":0.685167,"scheduling_latency_s":0.68518}1551vm-test-run-nixbot-gitlab> gitlab # [ 278.267349] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.643Z","message":"No existing merge request to be cleaned up."}1552vm-test-run-nixbot-gitlab> gitlab # [ 278.269280] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.652Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe0881d4557fa7f3e69581a4","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe0881d4557fa7f3e69581a4: done: 0.017676 sec","job_status":"done","queue_duration_s":0.685167,"scheduling_latency_s":0.68518,"redis_calls":10,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017676,"completed_at":"2026-06-29T21:59:05.652Z","load_balancing_strategy":"primary","db_duration_s":0.001459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [ 278.308614] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.653Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f2dbe21a42ba4c68802b1337","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f2dbe21a42ba4c68802b1337: start","job_status":"start","queue_duration_s":0.703949,"scheduling_latency_s":0.703962}1554vm-test-run-nixbot-gitlab> gitlab # [ 278.320970] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.663Z","message":"No existing merge request to be cleaned up."}1555vm-test-run-nixbot-gitlab> gitlab # [ 278.322637] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.670Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f2dbe21a42ba4c68802b1337","created_at":"2026-06-29T21:59:04.938Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-ea2b2a52eb864b95","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c7da599ac383bd7c63999ebaa237b52c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:04.949Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f2dbe21a42ba4c68802b1337: done: 0.017273 sec","job_status":"done","queue_duration_s":0.703949,"scheduling_latency_s":0.703962,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017273,"completed_at":"2026-06-29T21:59:05.670Z","load_balancing_strategy":"primary","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [ 278.362573] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bb8e6cbfe8bf73594fccd01","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bb8e6cbfe8bf73594fccd01: start","job_status":"start","queue_duration_s":0.436363,"scheduling_latency_s":0.436381}1557vm-test-run-nixbot-gitlab> gitlab # [ 278.375240] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bb8e6cbfe8bf73594fccd01","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bb8e6cbfe8bf73594fccd01: done: 0.014901 sec","job_status":"done","queue_duration_s":0.436363,"scheduling_latency_s":0.436381,"redis_calls":13,"redis_duration_s":0.0016409999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014901,"completed_at":"2026-06-29T21:59:05.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [ 278.414953] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fcd4085128a8217437c109","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fcd4085128a8217437c109: start","job_status":"start","queue_duration_s":0.452989,"scheduling_latency_s":0.453008}1559vm-test-run-nixbot-gitlab> gitlab # [ 278.428062] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fcd4085128a8217437c109","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fcd4085128a8217437c109: done: 0.03418 sec","job_status":"done","queue_duration_s":0.452989,"scheduling_latency_s":0.453008,"redis_calls":13,"redis_duration_s":0.002004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03418,"completed_at":"2026-06-29T21:59:05.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [ 278.468561] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ec1a7def33f3ffbd908eee","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ec1a7def33f3ffbd908eee: start","job_status":"start","queue_duration_s":0.488181,"scheduling_latency_s":0.488198}1561vm-test-run-nixbot-gitlab> gitlab # [ 278.481400] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ec1a7def33f3ffbd908eee","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ec1a7def33f3ffbd908eee: done: 0.014001 sec","job_status":"done","queue_duration_s":0.488181,"scheduling_latency_s":0.488198,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014001,"completed_at":"2026-06-29T21:59:05.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [ 278.520578] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5e8da0a82c60706a43dac22","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5e8da0a82c60706a43dac22: start","job_status":"start","queue_duration_s":0.503234,"scheduling_latency_s":0.503251}1563vm-test-run-nixbot-gitlab> gitlab # [ 278.533181] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5e8da0a82c60706a43dac22","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5e8da0a82c60706a43dac22: done: 0.013852 sec","job_status":"done","queue_duration_s":0.503234,"scheduling_latency_s":0.503251,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013852,"completed_at":"2026-06-29T21:59:05.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [ 278.574216] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bae80555c41edac0d55e4da","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bae80555c41edac0d55e4da: start","job_status":"start","queue_duration_s":0.518101,"scheduling_latency_s":0.518118}1565vm-test-run-nixbot-gitlab> gitlab # [ 278.587151] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bae80555c41edac0d55e4da","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bae80555c41edac0d55e4da: done: 0.013954 sec","job_status":"done","queue_duration_s":0.518101,"scheduling_latency_s":0.518118,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013954,"completed_at":"2026-06-29T21:59:05.767Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [ 278.628157] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02d38a43340de34ba65219fd","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02d38a43340de34ba65219fd: start","job_status":"start","queue_duration_s":0.533112,"scheduling_latency_s":0.533129}1567vm-test-run-nixbot-gitlab> gitlab # [ 278.640843] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02d38a43340de34ba65219fd","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02d38a43340de34ba65219fd: done: 0.01383 sec","job_status":"done","queue_duration_s":0.533112,"scheduling_latency_s":0.533129,"redis_calls":13,"redis_duration_s":0.0015570000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01383,"completed_at":"2026-06-29T21:59:05.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568vm-test-run-nixbot-gitlab> gitlab # [ 278.680540] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ce34d68d1bc4a4838693a6","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ce34d68d1bc4a4838693a6: start","job_status":"start","queue_duration_s":0.547976,"scheduling_latency_s":0.547992}1569vm-test-run-nixbot-gitlab> gitlab # [ 278.693352] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ce34d68d1bc4a4838693a6","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ce34d68d1bc4a4838693a6: done: 0.014015 sec","job_status":"done","queue_duration_s":0.547976,"scheduling_latency_s":0.547992,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014015,"completed_at":"2026-06-29T21:59:05.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [ 278.732652] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595189c41a4373505c41dc24","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595189c41a4373505c41dc24: start","job_status":"start","queue_duration_s":0.563061,"scheduling_latency_s":0.563078}1571vm-test-run-nixbot-gitlab> gitlab # [ 278.745175] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595189c41a4373505c41dc24","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595189c41a4373505c41dc24: done: 0.013984 sec","job_status":"done","queue_duration_s":0.563061,"scheduling_latency_s":0.563078,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013984,"completed_at":"2026-06-29T21:59:05.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [ 278.787888] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c9ecc0d82e2931d3f2180a3","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c9ecc0d82e2931d3f2180a3: start","job_status":"start","queue_duration_s":0.578443,"scheduling_latency_s":0.57846}1573vm-test-run-nixbot-gitlab> gitlab # [ 278.826695] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c9ecc0d82e2931d3f2180a3","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c9ecc0d82e2931d3f2180a3: done: 0.013783 sec","job_status":"done","queue_duration_s":0.578443,"scheduling_latency_s":0.57846,"redis_calls":13,"redis_duration_s":0.001874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013783,"completed_at":"2026-06-29T21:59:05.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574vm-test-run-nixbot-gitlab> gitlab # [ 278.899203] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98cdcdf53b0ac40bd1c1d007","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98cdcdf53b0ac40bd1c1d007: start","job_status":"start","queue_duration_s":0.593282,"scheduling_latency_s":0.593299}1575vm-test-run-nixbot-gitlab> gitlab # [ 278.911987] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98cdcdf53b0ac40bd1c1d007","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98cdcdf53b0ac40bd1c1d007: done: 0.013599 sec","job_status":"done","queue_duration_s":0.593282,"scheduling_latency_s":0.593299,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013599,"completed_at":"2026-06-29T21:59:05.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [ 278.952239] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e01819d25c9452c8cce141","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e01819d25c9452c8cce141: start","job_status":"start","queue_duration_s":0.607905,"scheduling_latency_s":0.607922}1577vm-test-run-nixbot-gitlab> gitlab # [ 278.966366] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e01819d25c9452c8cce141","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e01819d25c9452c8cce141: done: 0.013858 sec","job_status":"done","queue_duration_s":0.607905,"scheduling_latency_s":0.607922,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013858,"completed_at":"2026-06-29T21:59:05.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [ 279.006143] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b16f87706622dfb02cb3c6","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b16f87706622dfb02cb3c6: start","job_status":"start","queue_duration_s":0.622793,"scheduling_latency_s":0.622811}1579vm-test-run-nixbot-gitlab> gitlab # [ 279.019315] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b16f87706622dfb02cb3c6","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b16f87706622dfb02cb3c6: done: 0.013922 sec","job_status":"done","queue_duration_s":0.622793,"scheduling_latency_s":0.622811,"redis_calls":13,"redis_duration_s":0.001386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013922,"completed_at":"2026-06-29T21:59:05.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [ 279.059920] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ad259b217463a5775d13973","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ad259b217463a5775d13973: start","job_status":"start","queue_duration_s":0.637767,"scheduling_latency_s":0.637784}1581vm-test-run-nixbot-gitlab> gitlab # [ 279.073194] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ad259b217463a5775d13973","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ad259b217463a5775d13973: done: 0.014219 sec","job_status":"done","queue_duration_s":0.637767,"scheduling_latency_s":0.637784,"redis_calls":13,"redis_duration_s":0.001648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014219,"completed_at":"2026-06-29T21:59:05.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [ 279.112686] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ffded19e3680d0f1613043","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ffded19e3680d0f1613043: start","job_status":"start","queue_duration_s":0.653172,"scheduling_latency_s":0.653189}1583vm-test-run-nixbot-gitlab> gitlab # [ 279.125825] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ffded19e3680d0f1613043","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ffded19e3680d0f1613043: done: 0.014656 sec","job_status":"done","queue_duration_s":0.653172,"scheduling_latency_s":0.653189,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014656,"completed_at":"2026-06-29T21:59:05.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [ 279.167857] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c19a20c5e3fd9e55fac11992","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c19a20c5e3fd9e55fac11992: start","job_status":"start","queue_duration_s":0.668933,"scheduling_latency_s":0.668951}1585vm-test-run-nixbot-gitlab> gitlab # [ 279.181183] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c19a20c5e3fd9e55fac11992","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c19a20c5e3fd9e55fac11992: done: 0.013839 sec","job_status":"done","queue_duration_s":0.668933,"scheduling_latency_s":0.668951,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013839,"completed_at":"2026-06-29T21:59:05.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [ 279.223299] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7730d51ec27ce73e1f090f60","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7730d51ec27ce73e1f090f60: start","job_status":"start","queue_duration_s":0.684195,"scheduling_latency_s":0.684212}1587vm-test-run-nixbot-gitlab> gitlab # [ 279.236274] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7730d51ec27ce73e1f090f60","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7730d51ec27ce73e1f090f60: done: 0.013816 sec","job_status":"done","queue_duration_s":0.684195,"scheduling_latency_s":0.684212,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013816,"completed_at":"2026-06-29T21:59:05.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [ 279.276650] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f81921112d048a0f7a4ec59d","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f81921112d048a0f7a4ec59d: start","job_status":"start","queue_duration_s":0.699041,"scheduling_latency_s":0.699059}1589vm-test-run-nixbot-gitlab> gitlab # [ 279.289359] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f81921112d048a0f7a4ec59d","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f81921112d048a0f7a4ec59d: done: 0.013698 sec","job_status":"done","queue_duration_s":0.699041,"scheduling_latency_s":0.699059,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013698,"completed_at":"2026-06-29T21:59:05.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [ 279.330064] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab6f249db43040c47fb09c23","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab6f249db43040c47fb09c23: start","job_status":"start","queue_duration_s":0.71378,"scheduling_latency_s":0.713796}1591vm-test-run-nixbot-gitlab> gitlab # [ 279.343339] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab6f249db43040c47fb09c23","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab6f249db43040c47fb09c23: done: 0.013483 sec","job_status":"done","queue_duration_s":0.71378,"scheduling_latency_s":0.713796,"redis_calls":13,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013483,"completed_at":"2026-06-29T21:59:05.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [ 279.384907] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b81167228b4e7a5f4afb0f0","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b81167228b4e7a5f4afb0f0: start","job_status":"start","queue_duration_s":0.728576,"scheduling_latency_s":0.728594}1593vm-test-run-nixbot-gitlab> gitlab # [ 279.423173] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b81167228b4e7a5f4afb0f0","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b81167228b4e7a5f4afb0f0: done: 0.014352 sec","job_status":"done","queue_duration_s":0.728576,"scheduling_latency_s":0.728594,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014352,"completed_at":"2026-06-29T21:59:05.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [ 279.493619] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f53a6da23dfed0c7916c093","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f53a6da23dfed0c7916c093: start","job_status":"start","queue_duration_s":0.74395,"scheduling_latency_s":0.743966}1595vm-test-run-nixbot-gitlab> gitlab # [ 279.508181] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f53a6da23dfed0c7916c093","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f53a6da23dfed0c7916c093: done: 0.014294 sec","job_status":"done","queue_duration_s":0.74395,"scheduling_latency_s":0.743966,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014294,"completed_at":"2026-06-29T21:59:05.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [ 279.549270] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:05.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1940f9f7f3f7c75b4a04b38","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1940f9f7f3f7c75b4a04b38: start","job_status":"start","queue_duration_s":0.759266,"scheduling_latency_s":0.759282}1597vm-test-run-nixbot-gitlab> gitlab # [ 279.562132] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1940f9f7f3f7c75b4a04b38","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1940f9f7f3f7c75b4a04b38: done: 0.013686 sec","job_status":"done","queue_duration_s":0.759266,"scheduling_latency_s":0.759282,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013686,"completed_at":"2026-06-29T21:59:06.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [ 279.602760] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36328c4291c7bf55dfd291ef","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36328c4291c7bf55dfd291ef: start","job_status":"start","queue_duration_s":0.774361,"scheduling_latency_s":0.774378}1599vm-test-run-nixbot-gitlab> gitlab # [ 279.615601] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36328c4291c7bf55dfd291ef","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36328c4291c7bf55dfd291ef: done: 0.013832 sec","job_status":"done","queue_duration_s":0.774361,"scheduling_latency_s":0.774378,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013832,"completed_at":"2026-06-29T21:59:06.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [ 279.656173] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"476da03444dacfa542f79440","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-476da03444dacfa542f79440: start","job_status":"start","queue_duration_s":0.789284,"scheduling_latency_s":0.789301}1601vm-test-run-nixbot-gitlab> gitlab # [ 279.668933] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"476da03444dacfa542f79440","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-476da03444dacfa542f79440: done: 0.014857 sec","job_status":"done","queue_duration_s":0.789284,"scheduling_latency_s":0.789301,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014857,"completed_at":"2026-06-29T21:59:06.039Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [ 279.709719] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873edad01f108845ba18d60b","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873edad01f108845ba18d60b: start","job_status":"start","queue_duration_s":0.805196,"scheduling_latency_s":0.805235}1603vm-test-run-nixbot-gitlab> gitlab # [ 279.723037] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873edad01f108845ba18d60b","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873edad01f108845ba18d60b: done: 0.01436 sec","job_status":"done","queue_duration_s":0.805196,"scheduling_latency_s":0.805235,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01436,"completed_at":"2026-06-29T21:59:06.054Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [ 279.764260] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e815e9bed3f96796bcde9965","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e815e9bed3f96796bcde9965: start","job_status":"start","queue_duration_s":0.821119,"scheduling_latency_s":0.821138}1605vm-test-run-nixbot-gitlab> gitlab # [ 279.777502] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e815e9bed3f96796bcde9965","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e815e9bed3f96796bcde9965: done: 0.014468 sec","job_status":"done","queue_duration_s":0.821119,"scheduling_latency_s":0.821138,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014468,"completed_at":"2026-06-29T21:59:06.070Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606vm-test-run-nixbot-gitlab> gitlab # [ 279.820495] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87524b415cb8fdb535c7a3a1","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87524b415cb8fdb535c7a3a1: start","job_status":"start","queue_duration_s":0.836694,"scheduling_latency_s":0.836711}1607vm-test-run-nixbot-gitlab> gitlab # [ 279.833426] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87524b415cb8fdb535c7a3a1","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87524b415cb8fdb535c7a3a1: done: 0.013946 sec","job_status":"done","queue_duration_s":0.836694,"scheduling_latency_s":0.836711,"redis_calls":13,"redis_duration_s":0.001786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013946,"completed_at":"2026-06-29T21:59:06.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608vm-test-run-nixbot-gitlab> gitlab # [ 279.875048] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97ff0ddc5eda88dadda4d7fd","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97ff0ddc5eda88dadda4d7fd: start","job_status":"start","queue_duration_s":0.851774,"scheduling_latency_s":0.851791}1609vm-test-run-nixbot-gitlab> gitlab # [ 279.888181] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97ff0ddc5eda88dadda4d7fd","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97ff0ddc5eda88dadda4d7fd: done: 0.015575 sec","job_status":"done","queue_duration_s":0.851774,"scheduling_latency_s":0.851791,"redis_calls":13,"redis_duration_s":0.002023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-06-29T21:59:06.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [ 279.928779] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79455c4c23578d85eb200d78","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79455c4c23578d85eb200d78: start","job_status":"start","queue_duration_s":0.868774,"scheduling_latency_s":0.868793}1611vm-test-run-nixbot-gitlab> gitlab # [ 279.941829] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79455c4c23578d85eb200d78","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79455c4c23578d85eb200d78: done: 0.014371 sec","job_status":"done","queue_duration_s":0.868774,"scheduling_latency_s":0.868793,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014371,"completed_at":"2026-06-29T21:59:06.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [ 279.984147] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dec91cb5302677a52709f530","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dec91cb5302677a52709f530: start","job_status":"start","queue_duration_s":0.884203,"scheduling_latency_s":0.884219}1613vm-test-run-nixbot-gitlab> gitlab # [ 279.997931] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dec91cb5302677a52709f530","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dec91cb5302677a52709f530: done: 0.016608 sec","job_status":"done","queue_duration_s":0.884203,"scheduling_latency_s":0.884219,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016608,"completed_at":"2026-06-29T21:59:06.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614vm-test-run-nixbot-gitlab> gitlab # [ 280.039265] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e97e2794e7c5d8639f7a0e4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e97e2794e7c5d8639f7a0e4: start","job_status":"start","queue_duration_s":0.901895,"scheduling_latency_s":0.901912}1615vm-test-run-nixbot-gitlab> gitlab # [ 280.052632] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e97e2794e7c5d8639f7a0e4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e97e2794e7c5d8639f7a0e4: done: 0.014249 sec","job_status":"done","queue_duration_s":0.901895,"scheduling_latency_s":0.901912,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014249,"completed_at":"2026-06-29T21:59:06.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616vm-test-run-nixbot-gitlab> gitlab # [ 280.094201] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"673f32709433e733e6fc8d9c","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-673f32709433e733e6fc8d9c: start","job_status":"start","queue_duration_s":0.917289,"scheduling_latency_s":0.917307}1617vm-test-run-nixbot-gitlab> gitlab # [ 280.107492] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"673f32709433e733e6fc8d9c","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-673f32709433e733e6fc8d9c: done: 0.013889 sec","job_status":"done","queue_duration_s":0.917289,"scheduling_latency_s":0.917307,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013889,"completed_at":"2026-06-29T21:59:06.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618vm-test-run-nixbot-gitlab> gitlab # [ 280.147914] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6722b714e92c736537e6adb0","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6722b714e92c736537e6adb0: start","job_status":"start","queue_duration_s":0.932197,"scheduling_latency_s":0.932214}1619vm-test-run-nixbot-gitlab> gitlab # [ 280.161180] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6722b714e92c736537e6adb0","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6722b714e92c736537e6adb0: done: 0.013794 sec","job_status":"done","queue_duration_s":0.932197,"scheduling_latency_s":0.932214,"redis_calls":13,"redis_duration_s":0.0014130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013794,"completed_at":"2026-06-29T21:59:06.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [ 280.203056] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a4253f66389ccb6393cec2b","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a4253f66389ccb6393cec2b: start","job_status":"start","queue_duration_s":0.947072,"scheduling_latency_s":0.947089}1621vm-test-run-nixbot-gitlab> gitlab # [ 280.216220] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a4253f66389ccb6393cec2b","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a4253f66389ccb6393cec2b: done: 0.015642 sec","job_status":"done","queue_duration_s":0.947072,"scheduling_latency_s":0.947089,"redis_calls":13,"redis_duration_s":0.001815,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015642,"completed_at":"2026-06-29T21:59:06.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [ 280.257222] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37c74751ba668843e4e1ca25","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37c74751ba668843e4e1ca25: start","job_status":"start","queue_duration_s":0.963791,"scheduling_latency_s":0.963808}1623vm-test-run-nixbot-gitlab> gitlab # [ 280.270308] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37c74751ba668843e4e1ca25","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37c74751ba668843e4e1ca25: done: 0.014117 sec","job_status":"done","queue_duration_s":0.963791,"scheduling_latency_s":0.963808,"redis_calls":13,"redis_duration_s":0.0014390000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014117,"completed_at":"2026-06-29T21:59:06.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [ 280.311447] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2eabb7c72f1cc816107064d","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2eabb7c72f1cc816107064d: start","job_status":"start","queue_duration_s":0.979,"scheduling_latency_s":0.979017}1625vm-test-run-nixbot-gitlab> gitlab # [ 280.324116] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2eabb7c72f1cc816107064d","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2eabb7c72f1cc816107064d: done: 0.013764 sec","job_status":"done","queue_duration_s":0.979,"scheduling_latency_s":0.979017,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013764,"completed_at":"2026-06-29T21:59:06.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626vm-test-run-nixbot-gitlab> gitlab # [ 280.363671] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1596e83b24dafff2bacc8d92","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1596e83b24dafff2bacc8d92: start","job_status":"start","queue_duration_s":0.994229,"scheduling_latency_s":0.994246}1627vm-test-run-nixbot-gitlab> gitlab # [ 280.376338] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1596e83b24dafff2bacc8d92","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1596e83b24dafff2bacc8d92: done: 0.014298 sec","job_status":"done","queue_duration_s":0.994229,"scheduling_latency_s":0.994246,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014298,"completed_at":"2026-06-29T21:59:06.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628vm-test-run-nixbot-gitlab> gitlab # [ 280.418463] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29cd5caedafcc97d5feaf1f1","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29cd5caedafcc97d5feaf1f1: start","job_status":"start","queue_duration_s":1.009634,"scheduling_latency_s":1.009651}1629vm-test-run-nixbot-gitlab> gitlab # [ 280.431489] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29cd5caedafcc97d5feaf1f1","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29cd5caedafcc97d5feaf1f1: done: 0.01372 sec","job_status":"done","queue_duration_s":1.009634,"scheduling_latency_s":1.009651,"redis_calls":13,"redis_duration_s":0.001402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01372,"completed_at":"2026-06-29T21:59:06.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [ 280.471647] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81e8af840281e66826484085","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81e8af840281e66826484085: start","job_status":"start","queue_duration_s":1.024731,"scheduling_latency_s":1.024748}1631vm-test-run-nixbot-gitlab> gitlab # [ 280.484786] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81e8af840281e66826484085","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81e8af840281e66826484085: done: 0.013889 sec","job_status":"done","queue_duration_s":1.024731,"scheduling_latency_s":1.024748,"redis_calls":13,"redis_duration_s":0.0015840000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013889,"completed_at":"2026-06-29T21:59:06.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [ 280.525036] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47744842164d1ba372a1daed","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47744842164d1ba372a1daed: start","job_status":"start","queue_duration_s":1.039664,"scheduling_latency_s":1.039681}1633vm-test-run-nixbot-gitlab> gitlab # [ 280.537915] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47744842164d1ba372a1daed","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47744842164d1ba372a1daed: done: 0.013812 sec","job_status":"done","queue_duration_s":1.039664,"scheduling_latency_s":1.039681,"redis_calls":13,"redis_duration_s":0.001395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013812,"completed_at":"2026-06-29T21:59:06.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634vm-test-run-nixbot-gitlab> gitlab # [ 280.578976] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448350627ced8d0f10b49ed3","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448350627ced8d0f10b49ed3: start","job_status":"start","queue_duration_s":1.054537,"scheduling_latency_s":1.054553}1635vm-test-run-nixbot-gitlab> gitlab # [ 280.592083] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448350627ced8d0f10b49ed3","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448350627ced8d0f10b49ed3: done: 0.014628 sec","job_status":"done","queue_duration_s":1.054537,"scheduling_latency_s":1.054553,"redis_calls":13,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014628,"completed_at":"2026-06-29T21:59:06.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [ 280.632651] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28853d0c809159e53c1b3b88","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28853d0c809159e53c1b3b88: start","job_status":"start","queue_duration_s":1.070211,"scheduling_latency_s":1.070227}1637vm-test-run-nixbot-gitlab> gitlab # [ 280.645173] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28853d0c809159e53c1b3b88","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28853d0c809159e53c1b3b88: done: 0.013883 sec","job_status":"done","queue_duration_s":1.070211,"scheduling_latency_s":1.070227,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013883,"completed_at":"2026-06-29T21:59:06.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [ 280.685535] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27659851bd73dc48d3584482","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27659851bd73dc48d3584482: start","job_status":"start","queue_duration_s":1.085186,"scheduling_latency_s":1.085203}1639vm-test-run-nixbot-gitlab> gitlab # [ 280.698554] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27659851bd73dc48d3584482","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27659851bd73dc48d3584482: done: 0.013672 sec","job_status":"done","queue_duration_s":1.085186,"scheduling_latency_s":1.085203,"redis_calls":13,"redis_duration_s":0.0014160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013672,"completed_at":"2026-06-29T21:59:06.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [ 280.738297] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"645090b2c2d9752ad05bc05e","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-645090b2c2d9752ad05bc05e: start","job_status":"start","queue_duration_s":1.099948,"scheduling_latency_s":1.099964}1641vm-test-run-nixbot-gitlab> gitlab # [ 280.751000] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"645090b2c2d9752ad05bc05e","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-645090b2c2d9752ad05bc05e: done: 0.014448 sec","job_status":"done","queue_duration_s":1.099948,"scheduling_latency_s":1.099964,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014448,"completed_at":"2026-06-29T21:59:06.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [ 280.792040] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62061d7cfbb488cc14ee2034","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62061d7cfbb488cc14ee2034: start","job_status":"start","queue_duration_s":1.11548,"scheduling_latency_s":1.115496}1643vm-test-run-nixbot-gitlab> gitlab # [ 280.804873] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62061d7cfbb488cc14ee2034","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62061d7cfbb488cc14ee2034: done: 0.013802 sec","job_status":"done","queue_duration_s":1.11548,"scheduling_latency_s":1.115496,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013802,"completed_at":"2026-06-29T21:59:06.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [ 280.846322] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d13c8e9ac6bc61efa05e69b8","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d13c8e9ac6bc61efa05e69b8: start","job_status":"start","queue_duration_s":1.130302,"scheduling_latency_s":1.130318}1645vm-test-run-nixbot-gitlab> gitlab # [ 280.884214] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d13c8e9ac6bc61efa05e69b8","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d13c8e9ac6bc61efa05e69b8: done: 0.013524 sec","job_status":"done","queue_duration_s":1.130302,"scheduling_latency_s":1.130318,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013524,"completed_at":"2026-06-29T21:59:06.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [ 280.956064] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"797cb4274f28c1a60426bdc9","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-797cb4274f28c1a60426bdc9: start","job_status":"start","queue_duration_s":1.145251,"scheduling_latency_s":1.145268}1647vm-test-run-nixbot-gitlab> gitlab # [ 280.970782] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"797cb4274f28c1a60426bdc9","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-797cb4274f28c1a60426bdc9: done: 0.014421 sec","job_status":"done","queue_duration_s":1.145251,"scheduling_latency_s":1.145268,"redis_calls":13,"redis_duration_s":0.0019690000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014421,"completed_at":"2026-06-29T21:59:06.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [ 281.011539] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034466042c3116d46c20ec7e","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034466042c3116d46c20ec7e: start","job_status":"start","queue_duration_s":1.160827,"scheduling_latency_s":1.160844}1649vm-test-run-nixbot-gitlab> gitlab # [ 281.024728] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034466042c3116d46c20ec7e","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034466042c3116d46c20ec7e: done: 0.013926 sec","job_status":"done","queue_duration_s":1.160827,"scheduling_latency_s":1.160844,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013926,"completed_at":"2026-06-29T21:59:06.409Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [ 281.065048] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f6f4521fa6851bdafc14e4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f6f4521fa6851bdafc14e4: start","job_status":"start","queue_duration_s":1.175813,"scheduling_latency_s":1.17583}1651vm-test-run-nixbot-gitlab> gitlab # [ 281.078003] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f6f4521fa6851bdafc14e4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f6f4521fa6851bdafc14e4: done: 0.013728 sec","job_status":"done","queue_duration_s":1.175813,"scheduling_latency_s":1.17583,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013728,"completed_at":"2026-06-29T21:59:06.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [ 281.118041] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77541d3c0c8b8a9739becd31","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77541d3c0c8b8a9739becd31: start","job_status":"start","queue_duration_s":1.190603,"scheduling_latency_s":1.19062}1653vm-test-run-nixbot-gitlab> gitlab # [ 281.130813] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77541d3c0c8b8a9739becd31","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77541d3c0c8b8a9739becd31: done: 0.013853 sec","job_status":"done","queue_duration_s":1.190603,"scheduling_latency_s":1.19062,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013853,"completed_at":"2026-06-29T21:59:06.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [ 281.171720] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfd351e3a7686f434b347cb4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfd351e3a7686f434b347cb4: start","job_status":"start","queue_duration_s":1.205877,"scheduling_latency_s":1.205895}1655vm-test-run-nixbot-gitlab> gitlab # [ 281.184838] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:06.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfd351e3a7686f434b347cb4","created_at":"2026-06-29T21:59:05.159Z","trace_propagation_headers":{"sentry-trace":"727344c527f54a97b76a2e34b6dc8e97-5c18e4f94ea74afc","baggage":"sentry-trace_id=727344c527f54a97b76a2e34b6dc8e97,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5048b880a76c9428b5e83e6fe38fb106","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:05.235Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfd351e3a7686f434b347cb4: done: 0.013982 sec","job_status":"done","queue_duration_s":1.205877,"scheduling_latency_s":1.205895,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013982,"completed_at":"2026-06-29T21:59:06.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [ 310.128443] redis[1336]: 10 changes in 300 seconds. Saving...1657vm-test-run-nixbot-gitlab> gitlab # [ 310.135077] redis[1336]: Background saving started by pid 37951658vm-test-run-nixbot-gitlab> gitlab # [ 310.141380] redis[3795]: BGSAVE done, 343 keys saved, 0 keys skipped, 93625 bytes written.1659vm-test-run-nixbot-gitlab> gitlab # [ 310.145983] redis[3795]: DB saved on disk1660vm-test-run-nixbot-gitlab> gitlab # [ 310.148349] redis[3795]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1661vm-test-run-nixbot-gitlab> gitlab # [ 310.196240] runuser[3450]: pam_unix(runuser:session): session closed for user gitlab1662vm-test-run-nixbot-gitlab> gitlab # [ 310.200705] setup-gitlab-start[3799]: +++ date -d '+7 days' +%F1663vm-test-run-nixbot-gitlab> gitlab # [ 310.204449] setup-gitlab-start[3798]: ++ jq -r .token1664vm-test-run-nixbot-gitlab> gitlab # [ 310.207445] setup-gitlab-start[3797]: ++ curl -fs -X POST -H 'Authorization: Bearer 4c4aad8182ee4a044c738dfbdfa97f67cd027e23bbed093e57e463810a544e55' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-06"}' http://gitlab/api/v4/users/1/personal_access_tokens1665vm-test-run-nixbot-gitlab> gitlab # [ 310.229696] redis[1336]: Background saving terminated with success1666vm-test-run-nixbot-gitlab> gitlab # [ 310.494682] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:38.407Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a71a334bae84d1d5ebf3d785","created_at":"2026-06-29T21:59:38.372Z","trace_propagation_headers":{"sentry-trace":"c1cf2ff054814eba99856f42b9740048-60b0a433ea284715","baggage":"sentry-trace_id=c1cf2ff054814eba99856f42b9740048,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9CPHZNE0FCY9X8E1EJQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ef475886780ee700474351e3040db1b99c815ce31ef1d03703a16344453b2ebd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:38.403Z","job_size_bytes":451,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a71a334bae84d1d5ebf3d785: start","job_status":"start","queue_duration_s":0.003757,"scheduling_latency_s":0.003825}1667vm-test-run-nixbot-gitlab> gitlab # [ 310.543735] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:38 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2801668vm-test-run-nixbot-gitlab> gitlab # [ 310.549383] setup-gitlab-start[3408]: + TOKEN=glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid1669vm-test-run-nixbot-gitlab> gitlab # [ 310.552123] setup-gitlab-start[3408]: + test -n glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid1670vm-test-run-nixbot-gitlab> gitlab # [ 310.554621] setup-gitlab-start[3408]: + test glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid '!=' null1671vm-test-run-nixbot-gitlab> gitlab # [ 310.557325] setup-gitlab-start[3408]: + mkdir -p /var/lib/secrets1672vm-test-run-nixbot-gitlab> gitlab # [ 310.558957] setup-gitlab-start[3408]: + echo glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid1673vm-test-run-nixbot-gitlab> gitlab # [ 310.561255] setup-gitlab-start[3408]: + chmod 644 /var/lib/secrets/gitlab-token1674vm-test-run-nixbot-gitlab> gitlab # [ 310.563111] setup-gitlab-start[3408]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1675vm-test-run-nixbot-gitlab> gitlab # [ 310.807438] sidekiq-cluster[2518]: {"severity":"WARN","time":"2026-06-29T21:59:38.719Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a71a334bae84d1d5ebf3d785","created_at":"2026-06-29T21:59:38.372Z","trace_propagation_headers":{"sentry-trace":"c1cf2ff054814eba99856f42b9740048-60b0a433ea284715","baggage":"sentry-trace_id=c1cf2ff054814eba99856f42b9740048,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9CPHZNE0FCY9X8E1EJQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ef475886780ee700474351e3040db1b99c815ce31ef1d03703a16344453b2ebd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:38.403Z","job_size_bytes":451,"pid":2518,"sidekiq_tid":"lyoe","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a71a334bae84d1d5ebf3d785: fail: 0.310568 sec","job_status":"fail","queue_duration_s":0.003757,"scheduling_latency_s":0.003825,"redis_calls":17,"redis_duration_s":0.007921,"redis_read_bytes":651,"redis_write_bytes":1842,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003952,"redis_feature_flag_read_bytes":644,"redis_feature_flag_write_bytes":958,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002992,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.219389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00104842,"duration_s":0.310568,"completed_at":"2026-06-29T21:59:38.717Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.05486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1676vm-test-run-nixbot-gitlab> gitlab # [ 311.269637] gitaly[1444]: time="2026-06-29T21:59:39.136Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.135" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1677vm-test-run-nixbot-gitlab> gitlab # [ 311.279817] gitaly[1444]: time="2026-06-29T21:59:39.136Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.135" grpc.time_ms=0.683 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678vm-test-run-nixbot-gitlab> gitlab # [ 311.407842] gitaly[1444]: time="2026-06-29T21:59:39.322Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.192" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.322" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679vm-test-run-nixbot-gitlab> gitlab # [ 311.440647] gitaly[1444]: time="2026-06-29T21:59:39.323Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.192" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.322" grpc.time_ms=0.534 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680vm-test-run-nixbot-gitlab> gitlab # [ 311.473112] gitaly[1444]: time="2026-06-29T21:59:39.365Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK diskcache=1b66dec3-3382-4f36-9eb1-6100f5aba68f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:59:49.202" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.333" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681vm-test-run-nixbot-gitlab> gitlab # [ 311.493066] gitaly[1444]: time="2026-06-29T21:59:39.365Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=298 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:59:49.202" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.333" grpc.time_ms=32.596 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [ 311.513915] gitaly[1444]: time="2026-06-29T21:59:39.386Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.254" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.385" pid=1444 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1683vm-test-run-nixbot-gitlab> gitlab # [ 311.530106] gitaly[1444]: time="2026-06-29T21:59:39.386Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.254" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.385" grpc.time_ms=1.466 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1684vm-test-run-nixbot-gitlab> gitlab # [ 311.546966] gitaly[1444]: time="2026-06-29T21:59:39.412Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:49.264" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.395" grpc.time_ms=17.158 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685vm-test-run-nixbot-gitlab> gitlab # [ 311.567773] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:39.454Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2bb8885e0f8f159d13e6f960","created_at":"2026-06-29T21:59:39.428Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.451Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2bb8885e0f8f159d13e6f960: start","job_status":"start","queue_duration_s":0.002723,"scheduling_latency_s":0.002788}1686vm-test-run-nixbot-gitlab> gitlab # [ 311.586381] gitaly[1444]: time="2026-06-29T21:59:39.493Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.492" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687vm-test-run-nixbot-gitlab> gitlab # [ 311.598352] gitaly[1444]: time="2026-06-29T21:59:39.493Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.492" grpc.time_ms=0.496 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688vm-test-run-nixbot-gitlab> gitlab # [ 311.612656] gitaly[1444]: time="2026-06-29T21:59:39.526Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK diskcache=434614be-7958-4111-b765-dbdf1f8a8156 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:59:49.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.499" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689vm-test-run-nixbot-gitlab> gitlab # [ 311.624141] gitaly[1444]: time="2026-06-29T21:59:39.526Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:59:49.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.499" grpc.time_ms=27.1 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690vm-test-run-nixbot-gitlab> gitlab # [ 311.639342] gitaly[1444]: time="2026-06-29T21:59:39.535Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.044" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.534" pid=1444 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1691vm-test-run-nixbot-gitlab> gitlab # [ 311.651544] gitaly[1444]: time="2026-06-29T21:59:39.536Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.044" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.534" grpc.time_ms=1.324 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692vm-test-run-nixbot-gitlab> gitlab # [ 311.662909] gitaly[1444]: time="2026-06-29T21:59:39.561Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:49.052" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.542" grpc.time_ms=18.968 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693vm-test-run-nixbot-gitlab> gitlab # [ 311.676927] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:39.574Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2bb8885e0f8f159d13e6f960","created_at":"2026-06-29T21:59:39.428Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.451Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2bb8885e0f8f159d13e6f960: done: 0.120651 sec","job_status":"done","queue_duration_s":0.002723,"scheduling_latency_s":0.002788,"redis_calls":15,"redis_duration_s":0.007508,"redis_read_bytes":140,"redis_write_bytes":1876,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002506,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":711,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002915,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002087,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0574566,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.120651,"completed_at":"2026-06-29T21:59:39.574Z","load_balancing_strategy":"primary","db_duration_s":0.011029,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1694vm-test-run-nixbot-gitlab> gitlab # [ 311.724708] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:39.579Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a66f6324a5c3973d604be325","created_at":"2026-06-29T21:59:39.470Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.483Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a66f6324a5c3973d604be325: start","job_status":"start","queue_duration_s":0.095792,"scheduling_latency_s":0.095836}1695vm-test-run-nixbot-gitlab> gitlab # [ 311.739330] gitaly[1444]: time="2026-06-29T21:59:39.607Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:49.116" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.606" grpc.time_ms=0.272 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1696vm-test-run-nixbot-gitlab> gitlab # [ 311.750609] gitaly[1444]: time="2026-06-29T21:59:39.618Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:59:49.127" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:39.618" grpc.time_ms=0.635 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1697vm-test-run-nixbot-gitlab> gitlab # [ 311.861322] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:39.775Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a66f6324a5c3973d604be325","created_at":"2026-06-29T21:59:39.470Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.483Z","job_size_bytes":2,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a66f6324a5c3973d604be325: done: 0.196626 sec","job_status":"done","queue_duration_s":0.095792,"scheduling_latency_s":0.095836,"redis_calls":14,"redis_duration_s":0.0047799999999999995,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00113,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002028,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001622,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15203702,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.196626,"completed_at":"2026-06-29T21:59:39.775Z","load_balancing_strategy":"primary","db_duration_s":0.03775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1698vm-test-run-nixbot-gitlab> gitlab # [ 311.919863] postgres[1412]: [1412] LOG: checkpoint starting: time1699vm-test-run-nixbot-gitlab> gitlab # [ 312.050655] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:39.965Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"44d4996ff3849ecd79368fca","created_at":"2026-06-29T21:59:39.952Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.963Z","job_size_bytes":5,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-44d4996ff3849ecd79368fca: start","job_status":"start","queue_duration_s":0.001839,"scheduling_latency_s":0.00188}1700vm-test-run-nixbot-gitlab> gitlab # [ 312.181179] gitaly[1444]: time="2026-06-29T21:59:40.096Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPA9PVA3NYXJETDN533YCK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:50.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:40.084" grpc.time_ms=11.847 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701vm-test-run-nixbot-gitlab> gitlab # [ 312.220276] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:40.134Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"44d4996ff3849ecd79368fca","created_at":"2026-06-29T21:59:39.952Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:39.963Z","job_size_bytes":5,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-44d4996ff3849ecd79368fca: done: 0.16946 sec","job_status":"done","queue_duration_s":0.001839,"scheduling_latency_s":0.00188,"redis_calls":12,"redis_duration_s":0.0038599999999999997,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001285,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002002,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000573,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044738,"duration_s":0.16946,"completed_at":"2026-06-29T21:59:40.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030985,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1702vm-test-run-nixbot-gitlab> gitlab # [ 312.292106] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:40.137Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"92b20a0b5f96dbef37abea9e","created_at":"2026-06-29T21:59:40.025Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:40.036Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-92b20a0b5f96dbef37abea9e: start","job_status":"start","queue_duration_s":0.100468,"scheduling_latency_s":0.100502}1703vm-test-run-nixbot-gitlab> gitlab # [ 312.583826] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:40.497Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"92b20a0b5f96dbef37abea9e","created_at":"2026-06-29T21:59:40.025Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:40.036Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-92b20a0b5f96dbef37abea9e: done: 0.360115 sec","job_status":"done","queue_duration_s":0.100468,"scheduling_latency_s":0.100502,"redis_calls":11,"redis_duration_s":0.004145,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000967,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.304,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.304,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.260155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074204,"duration_s":0.360115,"completed_at":"2026-06-29T21:59:40.497Z","load_balancing_strategy":"primary","db_duration_s":0.111452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1704vm-test-run-nixbot-gitlab> gitlab # [ 312.675429] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:40.498Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0422f9ea0618dd658762415","created_at":"2026-06-29T21:59:40.044Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:40.055Z","job_size_bytes":89,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0422f9ea0618dd658762415: start","job_status":"start","queue_duration_s":0.443814,"scheduling_latency_s":0.443838}1705vm-test-run-nixbot-gitlab> gitlab # [ 312.704798] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:40.575Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0422f9ea0618dd658762415","created_at":"2026-06-29T21:59:40.044Z","trace_propagation_headers":{"sentry-trace":"493c2fa89f594f4184ddb93eff9759d2-378e77a55c1f4f3f","baggage":"sentry-trace_id=493c2fa89f594f4184ddb93eff9759d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPA9PVA3NYXJETDN533YCK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:40.055Z","job_size_bytes":89,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0422f9ea0618dd658762415: done: 0.076242 sec","job_status":"done","queue_duration_s":0.443814,"scheduling_latency_s":0.443838,"redis_calls":11,"redis_duration_s":0.0035009999999999998,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000859,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001919,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045686,"duration_s":0.076242,"completed_at":"2026-06-29T21:59:40.574Z","load_balancing_strategy":"primary","db_duration_s":0.027247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1706vm-test-run-nixbot-gitlab> gitlab # [ 312.770512] setup-gitlab-start[3408]: + mkdir -p /tmp/test-flake1707vm-test-run-nixbot-gitlab> gitlab # [ 312.771581] setup-gitlab-start[3408]: + cd /tmp/test-flake1708vm-test-run-nixbot-gitlab> gitlab # [ 312.772580] setup-gitlab-start[3408]: + git init -b master1709vm-test-run-nixbot-gitlab> gitlab # [ 312.773983] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:40 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 21181710vm-test-run-nixbot-gitlab> gitlab # [ 312.776313] setup-gitlab-start[3839]: Initialized empty Git repository in /tmp/test-flake/.git/1711vm-test-run-nixbot-gitlab> gitlab # [ 312.777806] setup-gitlab-start[3408]: + git config user.name test1712vm-test-run-nixbot-gitlab> gitlab # [ 312.778834] setup-gitlab-start[3408]: + git config user.email test@example.com1713vm-test-run-nixbot-gitlab> gitlab # [ 312.780065] setup-gitlab-start[3408]: + cat1714vm-test-run-nixbot-gitlab> gitlab # [ 312.780794] setup-gitlab-start[3408]: + git add flake.nix1715vm-test-run-nixbot-gitlab> gitlab # [ 312.781719] setup-gitlab-start[3408]: + git commit -m 'initial commit'1716vm-test-run-nixbot-gitlab> gitlab # [ 312.783001] setup-gitlab-start[3844]: [master (root-commit) 73944bd] initial commit1717vm-test-run-nixbot-gitlab> gitlab # [ 312.784266] setup-gitlab-start[3844]: 1 file changed, 10 insertions(+)1718vm-test-run-nixbot-gitlab> gitlab # [ 312.785321] setup-gitlab-start[3844]: create mode 100644 flake.nix1719vm-test-run-nixbot-gitlab> gitlab # [ 312.786326] setup-gitlab-start[3408]: + git remote add origin http://oauth2:glpat-bAzaDFGsduVMlrO7XfN6om86MQp1OjEH.01.0w1wm7nid@gitlab/root/test-flake.git1720vm-test-run-nixbot-gitlab> gitlab # [ 312.788427] setup-gitlab-start[3408]: + git push -u origin master1721vm-test-run-nixbot-gitlab> gitlab # [ 312.849190] gitaly[1444]: time="2026-06-29T21:59:40.763Z" level=info msg="burden monitor stats" pid=1444 top_10_by_cpu="[]" top_10_by_memory="[]"1722vm-test-run-nixbot-gitlab> gitlab # [ 312.901498] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 741723vm-test-run-nixbot-gitlab> gitlab # [ 313.074559] gitaly[1444]: time="2026-06-29T21:59:40.989Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC0JR72V0237G3SJGC1E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:40.968" grpc.time_ms=20.93 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1724vm-test-run-nixbot-gitlab> gitlab # [ 313.114084] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1721725vm-test-run-nixbot-gitlab> gitlab # [ 314.075827] gitaly[1444]: time="2026-06-29T21:59:41.989Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:51.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:41.970" grpc.time_ms=18.377 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726vm-test-run-nixbot-gitlab> gitlab # [ 314.202688] gitaly[1444]: time="2026-06-29T21:59:42.117Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T22:00:12.095" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T21:59:42.096" grpc.time_ms=21.558 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1727vm-test-run-nixbot-gitlab> gitlab # [ 314.325555] gitaly[1444]: time="2026-06-29T21:59:42.240Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 duration_ms="1.022µs" method=POST pid=1444 status=200 url="http://unix/api/v4/internal/allowed"1728vm-test-run-nixbot-gitlab> gitlab # [ 314.368404] gitaly[1444]: time="2026-06-29T21:59:42.283Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 duration_ms=41ns method=POST pid=1444 status=200 url="http://unix/api/v4/internal/pre_receive"1729vm-test-run-nixbot-gitlab> gitlab # [ 314.382297] gitaly[1444]: time="2026-06-29T21:59:42.284Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:41.217" grpc.time_ms=1066.961 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1730vm-test-run-nixbot-gitlab> gitlab # [ 314.469638] gitaly[1444]: time="2026-06-29T21:59:42.384Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:42.383" grpc.time_ms=1.49 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1731vm-test-run-nixbot-gitlab> gitlab # [ 314.564106] gitaly[1444]: time="2026-06-29T21:59:42.479Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:42.478" grpc.time_ms=0.906 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1732vm-test-run-nixbot-gitlab> gitlab # [ 314.651660] gitaly[1444]: time="2026-06-29T21:59:42.566Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:42.562" grpc.time_ms=3.747 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1733vm-test-run-nixbot-gitlab> gitlab # [ 314.885208] gitaly[1444]: time="2026-06-29T21:59:42.800Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:52.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:42.782" grpc.time_ms=17.369 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734vm-test-run-nixbot-gitlab> gitlab # [ 314.975540] gitaly[1444]: time="2026-06-29T21:59:42.890Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=32 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=0 command.real_time_ms=37 command.spawn_token_fork_ms=5 command.system_time_ms=24 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T21:59:52.066" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:42.847" grpc.time_ms=43.164 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1735vm-test-run-nixbot-gitlab> gitlab # [ 315.036293] gitaly[1444]: time="2026-06-29T21:59:42.947Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 duration_ms=325ns method=POST pid=1444 status=200 url="http://unix/api/v4/internal/post_receive"1736vm-test-run-nixbot-gitlab> gitlab # [ 315.042057] gitaly[1444]: time="2026-06-29T21:59:42.948Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:42.621" grpc.time_ms=327.162 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1737vm-test-run-nixbot-gitlab> gitlab # [ 315.062318] gitaly[1444]: time="2026-06-29T21:59:42.962Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 diskcache=806f8acf-a1a3-49de-a52c-18056ba0c4a1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:41.108" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1738vm-test-run-nixbot-gitlab> gitlab # [ 315.081307] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:42.973Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 73944bd72e96e739664970d1897e8de7fe8fda3b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"78c3180263f153347a59caee","created_at":"2026-06-29T21:59:42.682Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-8b288820f6fd4785","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:42.730Z","job_size_bytes":185,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-78c3180263f153347a59caee: start","job_status":"start","queue_duration_s":0.242623,"scheduling_latency_s":0.242652}1739vm-test-run-nixbot-gitlab> gitlab # [ 315.105100] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:42.993Z","message":"changes: \"0000000000000000000000000000000000000000 73944bd72e96e739664970d1897e8de7fe8fda3b refs/heads/master\\n\""}1740vm-test-run-nixbot-gitlab> gitlab # [ 315.108759] gitlab-workhorse[1369]: time="2026-06-29T21:59:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521741vm-test-run-nixbot-gitlab> gitlab # [ 315.111148] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:42 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 19561742vm-test-run-nixbot-gitlab> gitlab # [ 315.113934] systemd[1]: Finished setup-gitlab.service.1743vm-test-run-nixbot-gitlab> gitlab # [ 315.115039] setup-gitlab-start[3848]: To http://gitlab/root/test-flake.git1744vm-test-run-nixbot-gitlab> gitlab # [ 315.116208] setup-gitlab-start[3848]: * [new branch] master -> master1745vm-test-run-nixbot-gitlab> gitlab # [ 315.117309] setup-gitlab-start[3848]: branch 'master' set up to track 'origin/master'.1746vm-test-run-nixbot-gitlab> gitlab # [ 315.119055] gitaly[1444]: time="2026-06-29T21:59:42.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=388 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13565 command.oublock=32 command.real_time_ms=1845 command.spawn_token_fork_ms=6 command.system_time_ms=241 command.user_time_ms=147 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:41.108" grpc.time_ms=1854.272 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747vm-test-run-nixbot-gitlab> gitlab # [ 315.134157] systemd[1]: Starting nixbot CI service...1748vm-test-run-nixbot-gitlab> gitlab # [ 315.135103] gitaly[1444]: time="2026-06-29T21:59:43.040Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:59:53.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:43.028" grpc.time_ms=12.286 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749vm-test-run-nixbot-gitlab> gitlab # [ 315.149102] gitaly[1444]: time="2026-06-29T21:59:43.057Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:59:53.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:43.046" grpc.time_ms=11.169 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750vm-test-run-nixbot-gitlab> gitlab # [ 315.212244] gitaly[1444]: time="2026-06-29T21:59:43.127Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T21:59:53.086" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:43.107" grpc.time_ms=20.149 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751vm-test-run-nixbot-gitlab> gitlab # [ 315.290638] gitaly[1444]: time="2026-06-29T21:59:43.205Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:43.204" grpc.time_ms=0.827 pid=1444 span.kind=server system=grpc1752vm-test-run-nixbot-gitlab> gitlab # [ 315.349688] gitaly[1444]: time="2026-06-29T21:59:43.264Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:43.264" grpc.time_ms=0.592 pid=1444 span.kind=server system=grpc1753vm-test-run-nixbot-gitlab> gitlab # [ 315.374721] gitaly[1444]: time="2026-06-29T21:59:43.275Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 diskcache=0339e5bd-fa02-4811-9a19-360537ad3248 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:59:53.131" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:43.151" pid=1444 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [ 315.397602] gitaly[1444]: time="2026-06-29T21:59:43.275Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=109 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4857 command.oublock=8 command.real_time_ms=117 command.spawn_token_fork_ms=1 command.system_time_ms=57 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:59:53.131" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:43.151" grpc.time_ms=123.464 pid=1444 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1755vm-test-run-nixbot-gitlab> gitlab # [ 315.425688] gitaly[1444]: time="2026-06-29T21:59:43.301Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:59:53.280" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:43.301" grpc.time_ms=0.245 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756vm-test-run-nixbot-gitlab> gitlab # [ 315.444773] gitaly[1444]: time="2026-06-29T21:59:43.329Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T21:59:53.286" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:43.306" grpc.time_ms=22.173 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757vm-test-run-nixbot-gitlab> gitlab # [ 315.520093] gitaly[1444]: time="2026-06-29T21:59:43.435Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-29T22:00:13.027" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:43.427" grpc.time_ms=7.736 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [ 315.568656] gitaly[1444]: time="2026-06-29T21:59:43.468Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:59:53.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:43.467" grpc.time_ms=0.417 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759vm-test-run-nixbot-gitlab> gitlab # [ 315.597821] gitaly[1444]: time="2026-06-29T21:59:43.478Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T21:59:53.010" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T21:59:43.470" grpc.time_ms=7.962 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760vm-test-run-nixbot-gitlab> gitlab # [ 315.631032] gitaly[1444]: time="2026-06-29T21:59:43.509Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:59:53.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:43.495" grpc.time_ms=13.564 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761vm-test-run-nixbot-gitlab> gitlab # [ 315.658774] gitaly[1444]: time="2026-06-29T21:59:43.524Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:59:53.053" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:43.514" grpc.time_ms=10.088 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762vm-test-run-nixbot-gitlab> gitlab # [ 315.681869] gitaly[1444]: time="2026-06-29T21:59:43.559Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:00:13.135" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:43.535" grpc.time_ms=23.221 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 316.73 seconds)1764vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1765vm-test-run-nixbot-gitlab> gitlab # [ 316.106246] gitaly[1444]: time="2026-06-29T21:59:44.021Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T21:59:54.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.017" grpc.time_ms=3.311 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766vm-test-run-nixbot-gitlab> gitlab # [ 316.150952] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.064Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 73944bd72e96e739664970d1897e8de7fe8fda3b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"78c3180263f153347a59caee","created_at":"2026-06-29T21:59:42.682Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-8b288820f6fd4785","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:42.730Z","job_size_bytes":185,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-78c3180263f153347a59caee: done: 1.090437 sec","job_status":"done","queue_duration_s":0.242623,"scheduling_latency_s":0.242652,"gitaly_calls":13,"gitaly_duration_s":0.301422,"redis_calls":199,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.056136,"redis_read_bytes":957,"redis_write_bytes":19729,"redis_cache_calls":14,"redis_cache_duration_s":0.003124,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.008009,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.001793,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4935,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00492,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022929,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015361,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.32,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.355,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.075,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.652381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006458,"exclusive_lock_hold_duration_s":0.03792164,"duration_s":1.090437,"completed_at":"2026-06-29T21:59:44.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.117979,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1767vm-test-run-nixbot-gitlab> gitlab # [ 316.269723] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.067Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fc58f2d0d6c46e2950ee2e54","created_at":"2026-06-29T21:59:43.060Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.066Z","job_size_bytes":46,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fc58f2d0d6c46e2950ee2e54: start","job_status":"start","queue_duration_s":1.000911,"scheduling_latency_s":1.000954}1768vm-test-run-nixbot-gitlab> gitlab # [ 316.295194] gitaly[1444]: time="2026-06-29T21:59:44.162Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-30T03:59:44.150" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:44.151" grpc.time_ms=10.233 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769vm-test-run-nixbot-gitlab> gitlab # [ 316.312949] nixbot[3944]: {"ts": "2026-06-29T21:59:44.118Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1770vm-test-run-nixbot-gitlab> gitlab # [ 316.316714] nixbot[3944]: {"ts": "2026-06-29T21:59:44.141Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1771vm-test-run-nixbot-gitlab> gitlab # [ 316.319653] nixbot[3944]: {"ts": "2026-06-29T21:59:44.157Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1772vm-test-run-nixbot-gitlab> gitlab # [ 316.323011] nixbot[3944]: {"ts": "2026-06-29T21:59:44.161Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1773vm-test-run-nixbot-gitlab> gitlab # [ 316.326276] nixbot[3944]: {"ts": "2026-06-29T21:59:44.165Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1774vm-test-run-nixbot-gitlab> gitlab # [ 316.329264] nixbot[3944]: {"ts": "2026-06-29T21:59:44.171Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1775vm-test-run-nixbot-gitlab> gitlab # [ 316.332225] nixbot[3944]: {"ts": "2026-06-29T21:59:44.175Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1776vm-test-run-nixbot-gitlab> gitlab # [ 316.334987] nixbot[3944]: {"ts": "2026-06-29T21:59:44.177Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1777vm-test-run-nixbot-gitlab> gitlab # [ 316.337841] nixbot[3944]: {"ts": "2026-06-29T21:59:44.192Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1778vm-test-run-nixbot-gitlab> gitlab # [ 316.340590] nixbot[3944]: {"ts": "2026-06-29T21:59:44.195Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1779vm-test-run-nixbot-gitlab> gitlab # [ 316.343220] nixbot[3944]: {"ts": "2026-06-29T21:59:44.202Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1780vm-test-run-nixbot-gitlab> gitlab # [ 316.345832] nixbot[3944]: {"ts": "2026-06-29T21:59:44.212Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1781vm-test-run-nixbot-gitlab> gitlab # [ 316.349041] gitaly[1444]: time="2026-06-29T21:59:44.186Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T22:00:14.076" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.177" grpc.time_ms=8.795 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782vm-test-run-nixbot-gitlab> gitlab # [ 316.364406] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.231Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fc58f2d0d6c46e2950ee2e54","created_at":"2026-06-29T21:59:43.060Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.066Z","job_size_bytes":46,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fc58f2d0d6c46e2950ee2e54: done: 0.164835 sec","job_status":"done","queue_duration_s":1.000911,"scheduling_latency_s":1.000954,"gitaly_calls":2,"gitaly_duration_s":0.026624,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009428,"redis_read_bytes":68,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001935,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000939,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001846,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00378,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043992,"duration_s":0.164835,"completed_at":"2026-06-29T21:59:44.231Z","load_balancing_strategy":"primary","db_duration_s":0.015477,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1783vm-test-run-nixbot-gitlab> gitlab # [ 316.419760] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.234Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f4636b29e5451b4e1011cc48","created_at":"2026-06-29T21:59:43.085Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.097Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f4636b29e5451b4e1011cc48: start","job_status":"start","queue_duration_s":1.136741,"scheduling_latency_s":1.136774}1784vm-test-run-nixbot-gitlab> gitlab # [ 316.437814] nixbot[3944]: {"ts": "2026-06-29T21:59:44.220Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1785vm-test-run-nixbot-gitlab> gitlab # [ 316.439952] nixbot[3944]: {"ts": "2026-06-29T21:59:44.223Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1786vm-test-run-nixbot-gitlab> gitlab # [ 316.442362] nixbot[3944]: {"ts": "2026-06-29T21:59:44.230Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1787vm-test-run-nixbot-gitlab> gitlab # [ 316.444565] nixbot[3944]: {"ts": "2026-06-29T21:59:44.233Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1788vm-test-run-nixbot-gitlab> gitlab # [ 316.446608] nixbot[3944]: {"ts": "2026-06-29T21:59:44.236Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1789vm-test-run-nixbot-gitlab> gitlab # [ 316.448826] nixbot[3944]: {"ts": "2026-06-29T21:59:44.238Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1790vm-test-run-nixbot-gitlab> gitlab # [ 316.450821] nixbot[3944]: {"ts": "2026-06-29T21:59:44.243Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1791vm-test-run-nixbot-gitlab> gitlab # [ 316.452965] gitaly[1444]: time="2026-06-29T21:59:44.310Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 diskcache=9cecb7a4-18fa-4d47-abee-b80006d19357 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:59:44.280" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.280" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792vm-test-run-nixbot-gitlab> gitlab # [ 316.463097] gitaly[1444]: time="2026-06-29T21:59:44.311Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:59:44.280" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.280" grpc.time_ms=30.123 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793vm-test-run-nixbot-gitlab> gitlab # [ 316.479028] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.370Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f4636b29e5451b4e1011cc48","created_at":"2026-06-29T21:59:43.085Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.097Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f4636b29e5451b4e1011cc48: done: 0.136208 sec","job_status":"done","queue_duration_s":1.136741,"scheduling_latency_s":1.136774,"gitaly_calls":1,"gitaly_duration_s":0.034104,"redis_calls":15,"redis_duration_s":0.006593,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002552,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001745,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08880869999999999,"duration_s":0.136208,"completed_at":"2026-06-29T21:59:44.370Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1794vm-test-run-nixbot-gitlab> gitlab # [ 316.527847] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.373Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8d7d7b3a3fc338584a929573","created_at":"2026-06-29T21:59:43.294Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.297Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8d7d7b3a3fc338584a929573: start","job_status":"start","queue_duration_s":1.076338,"scheduling_latency_s":1.076383}1795vm-test-run-nixbot-gitlab> gitlab # [ 316.543995] gitaly[1444]: time="2026-06-29T21:59:44.445Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T22:00:14.324" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.425" grpc.time_ms=20.031 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1796vm-test-run-nixbot-gitlab> gitlab # [ 316.557124] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.461Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8d7d7b3a3fc338584a929573","created_at":"2026-06-29T21:59:43.294Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.297Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8d7d7b3a3fc338584a929573: done: 0.087499 sec","job_status":"done","queue_duration_s":1.076338,"scheduling_latency_s":1.076383,"gitaly_calls":1,"gitaly_duration_s":0.023305,"redis_calls":16,"redis_duration_s":0.0059499999999999996,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000472,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002379,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087499,"completed_at":"2026-06-29T21:59:44.461Z","load_balancing_strategy":"primary","db_duration_s":0.004202,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1797vm-test-run-nixbot-gitlab> gitlab # [ 316.605470] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:44.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f4c8f50cf3c0492e6103e123","created_at":"2026-06-29T21:59:43.963Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8529186342b9f00f9d643bf0f5d044302a0d26868311eb15163d9a18a46be228","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.969Z","job_size_bytes":467,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f4c8f50cf3c0492e6103e123: start","job_status":"start","queue_duration_s":0.494481,"scheduling_latency_s":0.494511}1798vm-test-run-nixbot-gitlab> gitlab # [ 316.643865] gitaly[1444]: time="2026-06-29T21:59:44.558Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:14.456" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.557" grpc.time_ms=1.017 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [ 316.665810] gitaly[1444]: time="2026-06-29T21:59:44.581Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T21:59:54.102" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:44.563" grpc.time_ms=17.254 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [ 316.810599] nixbot[3944]: INFO: Started server process [3944]1801vm-test-run-nixbot-gitlab> gitlab # [ 316.813604] nixbot[3944]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1802vm-test-run-nixbot-gitlab> gitlab # [ 316.821646] nixbot[3944]: INFO: ::ffff:127.0.0.1:41138 - "GET /health HTTP/1.1" 200 OK1803vm-test-run-nixbot-gitlab> gitlab # [ 316.831313] systemd[1]: Started nixbot CI service.1804vm-test-run-nixbot-gitlab> gitlab # [ 316.833662] systemd[1]: Reached target Multi-User System.1805vm-test-run-nixbot-gitlab> gitlab # [ 316.836328] systemd[1]: Startup finished in 744ms (kernel) + 5.042s (initrd) + 5min 11.045s (userspace) = 5min 16.833s.1806vm-test-run-nixbot-gitlab> gitlab # [ 316.885296] gitaly[1444]: time="2026-06-29T21:59:44.800Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:14.697" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.798" grpc.time_ms=1.93 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807vm-test-run-nixbot-gitlab> gitlab # [ 316.909747] gitaly[1444]: time="2026-06-29T21:59:44.811Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=73 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T22:00:14.705" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:44.806" grpc.time_ms=4.594 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808vm-test-run-nixbot-gitlab> gitlab # [ 317.065627] gitaly[1444]: time="2026-06-29T21:59:44.980Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:59:54.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:44.968" grpc.time_ms=11.684 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1809vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.11 seconds)1810vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1811vm-test-run-nixbot-gitlab> gitlab # [ 317.434794] nixbot[3944]: INFO: ::ffff:127.0.0.1:41152 - "GET /health HTTP/1.1" 200 OK1812vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1813vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1814vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1815vm-test-run-nixbot-gitlab> gitlab # [ 317.606626] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 1101816vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1817vm-test-run-nixbot-gitlab> []1818vm-test-run-nixbot-gitlab> gitlab # [ 317.749756] gitaly[1444]: time="2026-06-29T21:59:45.664Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=3 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAFTZNYDBPFGDWP24V86Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:00:15.637" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:45.638" grpc.time_ms=25.79 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819vm-test-run-nixbot-gitlab> gitlab # [ 317.795764] gitaly[1444]: time="2026-06-29T21:59:45.710Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=3 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:00:15.587" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:45.688" grpc.time_ms=22.691 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820vm-test-run-nixbot-gitlab> gitlab # [ 317.831654] gitaly[1444]: time="2026-06-29T21:59:45.746Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAC6F3S6Q8BP3FKT3JEF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T22:00:15.616" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:45.717" grpc.time_ms=29.774 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821vm-test-run-nixbot-gitlab> gitlab # [ 317.915307] sidekiq-cluster[2518]: {"severity":"WARN","time":"2026-06-29T21:59:45.829Z","class":"CreatePipelineWorker","correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"73944bd72e96e739664970d1897e8de7fe8fda3b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"73944bd72e96e739664970d1897e8de7fe8fda3b","ref":"refs/heads/master"},"retry":0}1822vm-test-run-nixbot-gitlab> gitlab # [ 317.937258] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:45.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f4c8f50cf3c0492e6103e123","created_at":"2026-06-29T21:59:43.963Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-cc85bf4120b34183","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8529186342b9f00f9d643bf0f5d044302a0d26868311eb15163d9a18a46be228","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:43.969Z","job_size_bytes":467,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f4c8f50cf3c0492e6103e123: done: 1.380061 sec","job_status":"done","queue_duration_s":0.494481,"scheduling_latency_s":0.494511,"gitaly_calls":7,"gitaly_duration_s":0.118105,"redis_calls":53,"redis_duration_s":0.019213,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001543,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.00917,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003827,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.142155,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00070676,"duration_s":1.380061,"completed_at":"2026-06-29T21:59:45.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.125361,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1823vm-test-run-nixbot-gitlab> gitlab # [ 318.020102] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:45.847Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1fc94b753d938e13d56178a1","created_at":"2026-06-29T21:59:44.198Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-f62b854d5a074da0","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:44.206Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1fc94b753d938e13d56178a1: start","job_status":"start","queue_duration_s":1.640402,"scheduling_latency_s":1.640442}1824vm-test-run-nixbot-gitlab> gitlab # [ 318.040628] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:45.930Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1fc94b753d938e13d56178a1","created_at":"2026-06-29T21:59:44.198Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-f62b854d5a074da0","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:44.206Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1fc94b753d938e13d56178a1: done: 0.083642 sec","job_status":"done","queue_duration_s":1.640402,"scheduling_latency_s":1.640442,"redis_calls":19,"redis_duration_s":0.0072840000000000005,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001559,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001041,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003333,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0136495,"duration_s":0.083642,"completed_at":"2026-06-29T21:59:45.930Z","load_balancing_strategy":"primary","db_duration_s":0.013216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1825vm-test-run-nixbot-gitlab> gitlab # [ 318.090285] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:45.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4fcf9fe176fba718b2709950","created_at":"2026-06-29T21:59:45.907Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-29014ff553d24cfe","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:45.916Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4fcf9fe176fba718b2709950: start","job_status":"start","queue_duration_s":0.016365,"scheduling_latency_s":0.016394}1826vm-test-run-nixbot-gitlab> gitlab # [ 318.143932] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:46.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4fcf9fe176fba718b2709950","created_at":"2026-06-29T21:59:45.907Z","trace_propagation_headers":{"sentry-trace":"6a5390fdd3d04766acd8a652578efabf-29014ff553d24cfe","baggage":"sentry-trace_id=6a5390fdd3d04766acd8a652578efabf,sentry-release=Unknown"},"correlation_id":"01KWAPAC6F3S6Q8BP3FKT3JEF5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:45.916Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4fcf9fe176fba718b2709950: done: 0.125831 sec","job_status":"done","queue_duration_s":0.016365,"scheduling_latency_s":0.016394,"redis_calls":11,"redis_duration_s":0.003714,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00232,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000668,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044596,"duration_s":0.125831,"completed_at":"2026-06-29T21:59:46.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1827vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1828vm-test-run-nixbot-gitlab> gitlab # [ 318.782932] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 1081829vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1830vm-test-run-nixbot-gitlab> []1831vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1832vm-test-run-nixbot-gitlab> gitlab # [ 319.955324] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 1101833vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1834vm-test-run-nixbot-gitlab> []1835vm-test-run-nixbot-gitlab> gitlab # [ 320.226371] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 34051836vm-test-run-nixbot-gitlab> gitlab # [ 320.229687] nixbot[3944]: {"ts": "2026-06-29T21:59:48.144Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1837vm-test-run-nixbot-gitlab> gitlab # [ 320.276049] nixbot[3944]: {"ts": "2026-06-29T21:59:48.191Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1838vm-test-run-nixbot-gitlab> gitlab # [ 320.388912] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1001839vm-test-run-nixbot-gitlab> gitlab # [ 320.395602] nixbot[3944]: {"ts": "2026-06-29T21:59:48.305Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1840vm-test-run-nixbot-gitlab> gitlab # [ 320.402808] nixbot[3944]: {"ts": "2026-06-29T21:59:48.307Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1841vm-test-run-nixbot-gitlab> gitlab # [ 320.840667] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 4451842vm-test-run-nixbot-gitlab> gitlab # [ 320.847352] nixbot[3944]: {"ts": "2026-06-29T21:59:48.757Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1843vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1844vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1845vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-29T21:59:48.736Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1846vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 3.70 seconds)1847vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1848vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1849vm-test-run-nixbot-gitlab> gitlab # [ 321.131830] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1101850vm-test-run-nixbot-gitlab> gitlab # [ 321.183160] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3241851vm-test-run-nixbot-gitlab> gitlab # [ 321.188085] nixbot[3944]: {"ts": "2026-06-29T21:59:49.100Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1852vm-test-run-nixbot-gitlab> gitlab # [ 321.289262] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 641853vm-test-run-nixbot-gitlab> gitlab # [ 321.399755] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 1931854vm-test-run-nixbot-gitlab> gitlab # [ 321.412759] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 1151855vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1856vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1857vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1858vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)[ 321.424339] gitaly[1444]: time="2026-06-29T21:59:49.324Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAM6R1ZCV17RB4GAKXY4E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:49.310" grpc.time_ms=13.371 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1859vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1860vm-test-run-nixbot-gitlab> gitlab # [ 321.460188] nixbot[3944]: {"ts": "2026-06-29T21:59:49.313Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1861vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 45.00 KiB/s, done.1862vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1863vm-test-run-nixbot-gitlab> gitlab # [ 321.540226] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 1131864vm-test-run-nixbot-gitlab> gitlab # [ 321.545846] nixbot[3944]: {"ts": "2026-06-29T21:59:49.453Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1865vm-test-run-nixbot-gitlab> gitlab # [ 321.657397] gitaly[1444]: time="2026-06-29T21:59:49.570Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMETXR8ZA6026CVAAZ22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-29T22:00:19.049" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:49.551" grpc.time_ms=17.088 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866vm-test-run-nixbot-gitlab> gitlab # [ 321.702198] gitaly[1444]: time="2026-06-29T21:59:49.617Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMETXR8ZA6026CVAAZ22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:19.114" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:49.615" grpc.time_ms=1.233 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867vm-test-run-nixbot-gitlab> gitlab # [ 321.729446] gitaly[1444]: time="2026-06-29T21:59:49.644Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMETXR8ZA6026CVAAZ22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T21:59:59.137" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:49.628" grpc.time_ms=16.108 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868vm-test-run-nixbot-gitlab> gitlab # [ 321.774059] gitaly[1444]: time="2026-06-29T21:59:49.689Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T21:59:59.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:49.677" grpc.time_ms=11.671 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab # [ 321.825723] gitaly[1444]: time="2026-06-29T21:59:49.740Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T22:00:19.728" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T21:59:49.728" grpc.time_ms=12.229 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870vm-test-run-nixbot-gitlab> gitlab # [ 321.852346] gitaly[1444]: time="2026-06-29T21:59:49.753Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 duration_ms=160ns method=POST pid=1444 status=200 url="http://unix/api/v4/internal/allowed"1871vm-test-run-nixbot-gitlab> gitlab # [ 321.856337] nixbot[3944]: {"ts": "2026-06-29T21:59:49.767Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1872vm-test-run-nixbot-gitlab> gitlab # [ 321.859799] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 2981873vm-test-run-nixbot-gitlab> gitlab # [ 321.883475] gitaly[1444]: time="2026-06-29T21:59:49.797Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 duration_ms=43ns method=POST pid=1444 status=200 url="http://unix/api/v4/internal/pre_receive"1874vm-test-run-nixbot-gitlab> gitlab # [ 321.888900] gitaly[1444]: time="2026-06-29T21:59:49.797Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:49.592" grpc.time_ms=205.473 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1875vm-test-run-nixbot-gitlab> gitlab # [ 321.923060] gitaly[1444]: time="2026-06-29T21:59:49.838Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:49.836" grpc.time_ms=1.031 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1876vm-test-run-nixbot-gitlab> gitlab # [ 321.971819] gitaly[1444]: time="2026-06-29T21:59:49.887Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:49.886" grpc.time_ms=0.597 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1877vm-test-run-nixbot-gitlab> gitlab # [ 321.988559] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1291878vm-test-run-nixbot-gitlab> gitlab # [ 321.992479] nixbot[3944]: {"ts": "2026-06-29T21:59:49.902Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1879vm-test-run-nixbot-gitlab> gitlab # [ 322.000200] nixbot[3944]: {"ts": "2026-06-29T21:59:49.912Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1880vm-test-run-nixbot-gitlab> gitlab # [ 322.010847] gitaly[1444]: time="2026-06-29T21:59:49.926Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:49.925" grpc.time_ms=0.62 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1881vm-test-run-nixbot-gitlab> gitlab # [ 322.151762] gitaly[1444]: time="2026-06-29T21:59:50.067Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=9 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMY4255FKFHHZ5AAQTB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.051" grpc.time_ms=15.716 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1882vm-test-run-nixbot-gitlab> gitlab # [ 322.172682] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1101883vm-test-run-nixbot-gitlab> gitlab # [ 322.175399] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.088Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","73944bd72e96e739664970d1897e8de7fe8fda3b 143c43e7a9f21ebec3698d7b35eda5ca9cef1512 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"af0fd9f16e7a4d86c5fea4d2","created_at":"2026-06-29T21:59:50.058Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-393af295aed34ac3","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.062Z","job_size_bytes":185,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-af0fd9f16e7a4d86c5fea4d2: start","job_status":"start","queue_duration_s":0.02586,"scheduling_latency_s":0.025892}1884vm-test-run-nixbot-gitlab> gitlab # [ 322.207579] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.122Z","message":"changes: \"73944bd72e96e739664970d1897e8de7fe8fda3b 143c43e7a9f21ebec3698d7b35eda5ca9cef1512 refs/heads/master\\n\""}1885vm-test-run-nixbot-gitlab> gitlab # [ 322.215196] gitaly[1444]: time="2026-06-29T21:59:50.130Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 duration_ms=155ns method=POST pid=1444 status=200 url="http://unix/api/v4/internal/post_receive"1886vm-test-run-nixbot-gitlab> gitlab # [ 322.223566] gitaly[1444]: time="2026-06-29T21:59:50.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:49.974" grpc.time_ms=156.837 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1887vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1888vm-test-run-nixbot-gitlab> gitlab # 73944bd..143c43e master -> master1889vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.12 seconds)1890vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1891vm-test-run-nixbot-gitlab> gitlab # [ 322.237565] gitaly[1444]: time="2026-06-29T21:59:50.142Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 diskcache=e5359857-7300-42e9-87ae-9651038dd379 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:49.509" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1892vm-test-run-nixbot-gitlab> gitlab # [ 322.248913] gitlab-workhorse[1369]: time="2026-06-29T21:59:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521893vm-test-run-nixbot-gitlab> gitlab # [ 322.250940] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7551894vm-test-run-nixbot-gitlab> gitlab # [ 322.253139] gitaly[1444]: time="2026-06-29T21:59:50.142Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=184 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=12878 command.oublock=32 command.real_time_ms=620 command.spawn_token_fork_ms=6 command.system_time_ms=126 command.user_time_ms=58 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:49.509" grpc.time_ms=632.728 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1895vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1896vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512'1897vm-test-run-nixbot-gitlab> gitlab # [ 322.268674] gitaly[1444]: time="2026-06-29T21:59:50.172Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:00.149" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.160" grpc.time_ms=12.084 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898vm-test-run-nixbot-gitlab> gitlab # [ 322.283787] gitaly[1444]: time="2026-06-29T21:59:50.190Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:00:00.169" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:50.179" grpc.time_ms=10.537 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899vm-test-run-nixbot-gitlab> gitlab # [ 322.298905] gitaly[1444]: time="2026-06-29T21:59:50.214Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN1N7Q32B4G333QVFNBW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.203" pid=1444 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1900vm-test-run-nixbot-gitlab> gitlab # [ 322.312435] gitaly[1444]: time="2026-06-29T21:59:50.215Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN1N7Q32B4G333QVFNBW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.203" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1901vm-test-run-nixbot-gitlab> gitlab # [ 322.327148] gitlab-workhorse[1369]: time="2026-06-29T21:59:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491902vm-test-run-nixbot-gitlab> gitlab # [ 322.329382] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1471903vm-test-run-nixbot-gitlab> gitlab # [ 322.331772] gitaly[1444]: time="2026-06-29T21:59:50.216Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN1N7Q32B4G333QVFNBW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.203" grpc.time_ms=12.737 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1904vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512', in 0.08 seconds)1905vm-test-run-nixbot-gitlab> []1906vm-test-run-nixbot-gitlab> gitlab # [ 322.348236] gitaly[1444]: time="2026-06-29T21:59:50.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T22:00:00.204" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.214" grpc.time_ms=11.349 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907vm-test-run-nixbot-gitlab> gitlab # [ 322.363145] nixbot[3944]: INFO: ::ffff:127.0.0.1:50182 - "GET /api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 200 OK1908vm-test-run-nixbot-gitlab> gitlab # [ 322.365321] gitaly[1444]: time="2026-06-29T21:59:50.245Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=598 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=7 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-29T22:00:20.132" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.233" grpc.time_ms=11.821 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909vm-test-run-nixbot-gitlab> gitlab # [ 322.381251] gitaly[1444]: time="2026-06-29T21:59:50.264Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T22:00:00.247" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T21:59:50.257" grpc.time_ms=6.843 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910vm-test-run-nixbot-gitlab> gitlab # [ 322.395283] gitaly[1444]: time="2026-06-29T21:59:50.286Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:00.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.273" grpc.time_ms=13.003 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911vm-test-run-nixbot-gitlab> gitlab # [ 322.413177] gitaly[1444]: time="2026-06-29T21:59:50.328Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN6BJG8J95FXW4BKSXQJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.303" pid=1444 remote_ip="2001:db8:1::1" request_sha=5bac210838e1bad73f6c75b5fb90b956137e3947 response_bytes=596 span.kind=server system=grpc1912vm-test-run-nixbot-gitlab> gitlab # [ 322.425518] gitaly[1444]: time="2026-06-29T21:59:50.328Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1140 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN6BJG8J95FXW4BKSXQJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.303" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1913vm-test-run-nixbot-gitlab> gitlab # [ 322.441125] gitlab-workhorse[1369]: time="2026-06-29T21:59:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=5961914vm-test-run-nixbot-gitlab> gitlab # [ 322.443240] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 596 "" "git/2.54.0" 1091915vm-test-run-nixbot-gitlab> gitlab # [ 322.445371] gitaly[1444]: time="2026-06-29T21:59:50.328Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1140 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAN6BJG8J95FXW4BKSXQJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.303" grpc.time_ms=25.221 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1916vm-test-run-nixbot-gitlab> gitlab # [ 322.532256] gitaly[1444]: time="2026-06-29T21:59:50.447Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPANBAGFTYT7M8QV1HC80M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.434" grpc.time_ms=13.209 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1917vm-test-run-nixbot-gitlab> gitlab # [ 322.561978] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 691918vm-test-run-nixbot-gitlab> gitlab # [ 322.596538] gitaly[1444]: time="2026-06-29T21:59:50.511Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANDH7XERA3FTMPF2MEJR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.500" pid=1444 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1919vm-test-run-nixbot-gitlab> gitlab # [ 322.621414] gitaly[1444]: time="2026-06-29T21:59:50.512Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T22:00:00.049" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.510" grpc.time_ms=1.68 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920vm-test-run-nixbot-gitlab> gitlab # [ 322.646805] gitlab-workhorse[1369]: time="2026-06-29T21:59:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671921vm-test-run-nixbot-gitlab> gitlab # [ 322.650515] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 631922vm-test-run-nixbot-gitlab> gitlab # [ 322.654179] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.538Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","73944bd72e96e739664970d1897e8de7fe8fda3b 143c43e7a9f21ebec3698d7b35eda5ca9cef1512 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"af0fd9f16e7a4d86c5fea4d2","created_at":"2026-06-29T21:59:50.058Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-393af295aed34ac3","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.062Z","job_size_bytes":185,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-af0fd9f16e7a4d86c5fea4d2: done: 0.44973 sec","job_status":"done","queue_duration_s":0.02586,"scheduling_latency_s":0.025892,"gitaly_calls":7,"gitaly_duration_s":0.095838,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.021406,"redis_read_bytes":350,"redis_write_bytes":13067,"redis_cache_calls":9,"redis_cache_duration_s":0.001447,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.003359,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6767,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002344,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.00554,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008271,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.026,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.258439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00093886,"exclusive_lock_hold_duration_s":0.0212318,"duration_s":0.44973,"completed_at":"2026-06-29T21:59:50.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055631,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1923vm-test-run-nixbot-gitlab> gitlab # [ 322.724373] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.540Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"dad4e561f516af6f1232c59d","created_at":"2026-06-29T21:59:50.192Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.194Z","job_size_bytes":46,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-dad4e561f516af6f1232c59d: start","job_status":"start","queue_duration_s":0.345692,"scheduling_latency_s":0.345719}1924vm-test-run-nixbot-gitlab> gitlab # [ 322.742194] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.562Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"dad4e561f516af6f1232c59d","created_at":"2026-06-29T21:59:50.192Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.194Z","job_size_bytes":46,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-dad4e561f516af6f1232c59d: done: 0.021809 sec","job_status":"done","queue_duration_s":0.345692,"scheduling_latency_s":0.345719,"redis_calls":9,"redis_duration_s":0.0017339999999999999,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000171,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021809,"completed_at":"2026-06-29T21:59:50.561Z","load_balancing_strategy":"primary","db_duration_s":0.001723,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1925vm-test-run-nixbot-gitlab> gitlab # [ 322.788236] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.563Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"faaddc7f0cdaa43dedc0722f","created_at":"2026-06-29T21:59:50.208Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.210Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-faaddc7f0cdaa43dedc0722f: start","job_status":"start","queue_duration_s":0.353042,"scheduling_latency_s":0.35306}1926vm-test-run-nixbot-gitlab> gitlab # [ 322.805079] gitaly[1444]: time="2026-06-29T21:59:50.512Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANDH7XERA3FTMPF2MEJR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.500" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1927vm-test-run-nixbot-gitlab> gitlab # [ 322.819304] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.628Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"faaddc7f0cdaa43dedc0722f","created_at":"2026-06-29T21:59:50.208Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.210Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-faaddc7f0cdaa43dedc0722f: done: 0.065423 sec","job_status":"done","queue_duration_s":0.353042,"scheduling_latency_s":0.35306,"gitaly_calls":1,"gitaly_duration_s":0.031118,"redis_calls":12,"redis_duration_s":0.003865,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001795,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0475605,"duration_s":0.065423,"completed_at":"2026-06-29T21:59:50.628Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1928vm-test-run-nixbot-gitlab> gitlab # [ 322.865284] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:50.634Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2ff2f64a61acdd98924d445e","created_at":"2026-06-29T21:59:50.338Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:762d179e2297d1970b7a980b67c514deddc66837681c846aaf3c4731a30f248c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.340Z","job_size_bytes":467,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2ff2f64a61acdd98924d445e: start","job_status":"start","queue_duration_s":0.294319,"scheduling_latency_s":0.294346}1929vm-test-run-nixbot-gitlab> gitlab # [ 322.883234] gitaly[1444]: time="2026-06-29T21:59:50.512Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANDH7XERA3FTMPF2MEJR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:50.500" grpc.time_ms=12.112 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1930vm-test-run-nixbot-gitlab> gitlab # [ 322.898310] gitaly[1444]: time="2026-06-29T21:59:50.611Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 diskcache=bda7183a-8265-4336-8f24-df772d8686a6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:59:50.581" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.582" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [ 322.908326] gitaly[1444]: time="2026-06-29T21:59:50.611Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1512 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:59:50.581" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.582" grpc.time_ms=29.349 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [ 322.923932] gitaly[1444]: time="2026-06-29T21:59:50.616Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:20.613" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.613" grpc.time_ms=2.3 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933vm-test-run-nixbot-gitlab> gitlab # [ 322.937209] gitaly[1444]: time="2026-06-29T21:59:50.662Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:20.561" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.661" grpc.time_ms=0.631 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [ 322.950160] gitaly[1444]: time="2026-06-29T21:59:50.663Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T22:00:20.651" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.652" grpc.time_ms=11.044 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [ 322.965425] gitaly[1444]: time="2026-06-29T21:59:50.674Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:00:00.204" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.665" grpc.time_ms=9.381 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936vm-test-run-nixbot-gitlab> gitlab # [ 322.978985] gitaly[1444]: time="2026-06-29T21:59:50.682Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:00.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.672" grpc.time_ms=9.454 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [ 322.992823] gitaly[1444]: time="2026-06-29T21:59:50.714Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:20.613" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.713" grpc.time_ms=0.75 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938vm-test-run-nixbot-gitlab> gitlab # [ 323.006025] gitaly[1444]: time="2026-06-29T21:59:50.725Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=298 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T22:00:20.618" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.718" grpc.time_ms=6.669 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939vm-test-run-nixbot-gitlab> gitlab # [ 323.022914] gitaly[1444]: time="2026-06-29T21:59:50.754Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:00:00.083" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:50.754" grpc.time_ms=0.419 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1940vm-test-run-nixbot-gitlab> gitlab # [ 323.034464] gitaly[1444]: time="2026-06-29T21:59:50.775Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:00:00.093" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.764" grpc.time_ms=10.838 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941vm-test-run-nixbot-gitlab> gitlab # [ 323.048454] gitaly[1444]: time="2026-06-29T21:59:50.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:00.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:50.775" grpc.time_ms=10.674 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1942vm-test-run-nixbot-gitlab> gitlab # [ 323.062050] gitaly[1444]: time="2026-06-29T21:59:50.829Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:50.829" grpc.time_ms=0.532 pid=1444 span.kind=server system=grpc1943vm-test-run-nixbot-gitlab> gitlab # [ 323.072249] gitaly[1444]: time="2026-06-29T21:59:50.856Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:50.856" grpc.time_ms=0.454 pid=1444 span.kind=server system=grpc1944vm-test-run-nixbot-gitlab> gitlab # [ 323.082235] gitaly[1444]: time="2026-06-29T21:59:50.866Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 diskcache=708938a8-43f1-4aad-bb77-0b3f4ce44899 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:00:00.107" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:50.778" pid=1444 ref=refs/keep-around/73944bd72e96e739664970d1897e8de7fe8fda3b remote_ip="2001:db8:1::1" revision=73944bd72e96e739664970d1897e8de7fe8fda3b span.kind=server system=grpc username=root1945vm-test-run-nixbot-gitlab> gitlab # [ 323.094199] gitaly[1444]: time="2026-06-29T21:59:50.866Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5417 command.oublock=8 command.real_time_ms=82 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:00:00.107" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:50.778" grpc.time_ms=88.482 pid=1444 ref=refs/keep-around/73944bd72e96e739664970d1897e8de7fe8fda3b remote_ip="2001:db8:1::1" revision=73944bd72e96e739664970d1897e8de7fe8fda3b span.kind=server system=grpc username=root1946vm-test-run-nixbot-gitlab> gitlab # [ 323.109799] gitaly[1444]: time="2026-06-29T21:59:50.870Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:20.869" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:50.869" grpc.time_ms=0.494 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947vm-test-run-nixbot-gitlab> gitlab # [ 323.332288] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 6911948vm-test-run-nixbot-gitlab> gitlab # [ 323.341610] nixbot[3944]: {"ts": "2026-06-29T21:59:51.247Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b \"HTTP/1.1 201 Created\""}1949vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512'1950vm-test-run-nixbot-gitlab> gitlab # [ 323.350484] nixbot[3944]: {"ts": "2026-06-29T21:59:51.260Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-73944bd72e96-5ddea765", "argv0": "bwrap"}1951vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512', in 0.08 seconds)1952vm-test-run-nixbot-gitlab> []1953vm-test-run-nixbot-gitlab> gitlab # [ 323.407461] gitaly[1444]: time="2026-06-29T21:59:51.322Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=4 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:00:21.191" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:51.292" grpc.time_ms=30.299 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954vm-test-run-nixbot-gitlab> gitlab # [ 323.454249] gitaly[1444]: time="2026-06-29T21:59:51.355Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=10 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAMCB2ZZVMAHQK1XA1DK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T22:00:21.226" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:51.327" grpc.time_ms=27.489 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955vm-test-run-nixbot-gitlab> gitlab # [ 323.480066] nixbot[3944]: INFO: ::ffff:127.0.0.1:50186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 200 OK1956vm-test-run-nixbot-gitlab> gitlab # [ 323.483930] sidekiq-cluster[2518]: {"severity":"WARN","time":"2026-06-29T21:59:51.376Z","class":"CreatePipelineWorker","correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"73944bd72e96e739664970d1897e8de7fe8fda3b","after":"143c43e7a9f21ebec3698d7b35eda5ca9cef1512","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"143c43e7a9f21ebec3698d7b35eda5ca9cef1512","ref":"refs/heads/master"},"retry":0}1957vm-test-run-nixbot-gitlab> gitlab # [ 323.494800] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.384Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2ff2f64a61acdd98924d445e","created_at":"2026-06-29T21:59:50.338Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:762d179e2297d1970b7a980b67c514deddc66837681c846aaf3c4731a30f248c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.340Z","job_size_bytes":467,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2ff2f64a61acdd98924d445e: done: 0.749539 sec","job_status":"done","queue_duration_s":0.294319,"scheduling_latency_s":0.294346,"gitaly_calls":7,"gitaly_duration_s":0.105136,"redis_calls":33,"redis_duration_s":0.008765,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001248,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002739,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003791,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.609128,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.749539,"completed_at":"2026-06-29T21:59:51.384Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033975,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1958vm-test-run-nixbot-gitlab> gitlab # [ 323.548462] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.386Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"af69b104-345c-4a1f-85bb-0ef06dc21561\"}"],"class":"WebHookWorker","jid":"38f6765a1ccc42037bff68d9","created_at":"2026-06-29T21:59:50.487Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae3610512b9983df9e57889ab976a23d08f55b67631e480b6d0706661eab9da6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.493Z","job_size_bytes":1749,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-38f6765a1ccc42037bff68d9: start","job_status":"start","queue_duration_s":0.893048,"scheduling_latency_s":0.893083}1959vm-test-run-nixbot-gitlab> gitlab # [ 323.566944] systemd[1]: Started Nix Daemon.1960vm-test-run-nixbot-gitlab> gitlab # [ 323.666170] nix-daemon[4269]: accepted connection from pid 4264, user nixbot1961vm-test-run-nixbot-gitlab> gitlab # [ 323.669541] nix-daemon[4269]: accepted connection from pid 4265, user nixbot1962vm-test-run-nixbot-gitlab> gitlab # [ 323.726010] nixbot[3944]: INFO: ::ffff:127.0.0.1:50188 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1963vm-test-run-nixbot-gitlab> gitlab # [ 323.744406] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.659Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"af69b104-345c-4a1f-85bb-0ef06dc21561\"}"],"class":"WebHookWorker","jid":"38f6765a1ccc42037bff68d9","created_at":"2026-06-29T21:59:50.487Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-4dbabbbec42b4b14","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae3610512b9983df9e57889ab976a23d08f55b67631e480b6d0706661eab9da6","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.493Z","job_size_bytes":1749,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-38f6765a1ccc42037bff68d9: done: 0.272295 sec","job_status":"done","queue_duration_s":0.893048,"scheduling_latency_s":0.893083,"redis_calls":19,"redis_duration_s":0.004749,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001033,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000439,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002297,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.015822319999983847,"cpu_s":0.228439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037218,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.272295,"completed_at":"2026-06-29T21:59:51.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1964vm-test-run-nixbot-gitlab> gitlab # [ 323.856355] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.663Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"874037eaf159d3b1330b30c2","created_at":"2026-06-29T21:59:50.879Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.885Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-874037eaf159d3b1330b30c2: start","job_status":"start","queue_duration_s":0.77865,"scheduling_latency_s":0.778674}1965vm-test-run-nixbot-gitlab> gitlab # [ 323.879230] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.752Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"874037eaf159d3b1330b30c2","created_at":"2026-06-29T21:59:50.879Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.885Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-874037eaf159d3b1330b30c2: done: 0.088613 sec","job_status":"done","queue_duration_s":0.77865,"scheduling_latency_s":0.778674,"redis_calls":18,"redis_duration_s":0.005114,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002386,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054532,"duration_s":0.088613,"completed_at":"2026-06-29T21:59:51.752Z","load_balancing_strategy":"primary","db_duration_s":0.016246,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1966vm-test-run-nixbot-gitlab> gitlab # [ 323.926921] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.755Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"77a6bb0b19c7c83411b56888","created_at":"2026-06-29T21:59:50.886Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.890Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-77a6bb0b19c7c83411b56888: start","job_status":"start","queue_duration_s":0.864504,"scheduling_latency_s":0.864542}1967vm-test-run-nixbot-gitlab> gitlab # [ 323.945423] gitaly[1444]: time="2026-06-29T21:59:51.804Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=7 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPMK7RHGW7YG1KJJDMC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:51.778" grpc.time_ms=25.574 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1968vm-test-run-nixbot-gitlab> gitlab # [ 323.959065] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1061969vm-test-run-nixbot-gitlab> gitlab # [ 323.965941] gitaly[1444]: time="2026-06-29T21:59:51.881Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPR08RCTKGQKG2YJP9KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:51.869" pid=1444 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1970vm-test-run-nixbot-gitlab> gitlab # [ 323.977078] gitaly[1444]: time="2026-06-29T21:59:51.881Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPR08RCTKGQKG2YJP9KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:51.869" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1971vm-test-run-nixbot-gitlab> gitlab # [ 323.991202] gitlab-workhorse[1369]: time="2026-06-29T21:59:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671972vm-test-run-nixbot-gitlab> gitlab # [ 323.993394] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 731973vm-test-run-nixbot-gitlab> gitlab # [ 323.995522] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.891Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"77a6bb0b19c7c83411b56888","created_at":"2026-06-29T21:59:50.886Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:50.890Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-77a6bb0b19c7c83411b56888: done: 0.136721 sec","job_status":"done","queue_duration_s":0.864504,"scheduling_latency_s":0.864542,"redis_calls":21,"redis_duration_s":0.006577,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002409,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001279,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068004,"duration_s":0.136721,"completed_at":"2026-06-29T21:59:51.891Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1974vm-test-run-nixbot-gitlab> gitlab # [ 324.044548] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:51.895Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d97059b88bb650e43be4f610","created_at":"2026-06-29T21:59:51.194Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:51.201Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d97059b88bb650e43be4f610: start","job_status":"start","queue_duration_s":0.693783,"scheduling_latency_s":0.693811}1975vm-test-run-nixbot-gitlab> gitlab # [ 324.060682] gitaly[1444]: time="2026-06-29T21:59:51.881Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPR08RCTKGQKG2YJP9KF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:51.869" grpc.time_ms=11.611 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976vm-test-run-nixbot-gitlab> gitlab # [ 324.075811] nix-daemon[4274]: unexpected Nix daemon error: error: interrupted by the user1977vm-test-run-nixbot-gitlab> gitlab # [ 324.077224] nixbot[3944]: {"ts": "2026-06-29T21:59:51.919Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1978vm-test-run-nixbot-gitlab> gitlab # [ 324.079320] gitaly[1444]: time="2026-06-29T21:59:51.992Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:21.090" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:51.990" grpc.time_ms=0.964 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979vm-test-run-nixbot-gitlab> gitlab # [ 324.101261] gitaly[1444]: time="2026-06-29T21:59:52.016Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T22:00:22.004" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.005" grpc.time_ms=11.267 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980vm-test-run-nixbot-gitlab> gitlab # [ 324.117794] gitaly[1444]: time="2026-06-29T21:59:52.033Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:22.031" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.032" grpc.time_ms=0.811 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981vm-test-run-nixbot-gitlab> gitlab # [ 324.131866] gitaly[1444]: time="2026-06-29T21:59:52.034Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:02.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.023" grpc.time_ms=10.761 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [ 324.146785] gitaly[1444]: time="2026-06-29T21:59:52.060Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:00:02.039" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:52.059" grpc.time_ms=0.325 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983vm-test-run-nixbot-gitlab> gitlab # [ 324.158658] gitaly[1444]: time="2026-06-29T21:59:52.073Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:00:02.044" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.064" grpc.time_ms=9.028 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984vm-test-run-nixbot-gitlab> gitlab # [ 324.176750] gitaly[1444]: time="2026-06-29T21:59:52.092Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.090" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.091" grpc.time_ms=0.701 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985vm-test-run-nixbot-gitlab> gitlab # [ 324.200746] gitaly[1444]: time="2026-06-29T21:59:52.116Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:52.115" grpc.time_ms=0.468 pid=1444 span.kind=server system=grpc1986vm-test-run-nixbot-gitlab> gitlab # [ 324.212841] gitaly[1444]: time="2026-06-29T21:59:52.128Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPANGTHH6BZN2MYXVEV5A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.026" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.126" grpc.time_ms=0.792 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [ 324.228410] gitaly[1444]: time="2026-06-29T21:59:52.135Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.032" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.132" grpc.time_ms=2.299 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [ 324.243511] gitaly[1444]: time="2026-06-29T21:59:52.154Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:52.153" grpc.time_ms=0.847 pid=1444 span.kind=server system=grpc1989vm-test-run-nixbot-gitlab> gitlab # [ 324.259030] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.144Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d97059b88bb650e43be4f610","created_at":"2026-06-29T21:59:51.194Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-7ab36fb0afe34c71","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:51.201Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d97059b88bb650e43be4f610: done: 0.249308 sec","job_status":"done","queue_duration_s":0.693783,"scheduling_latency_s":0.693811,"gitaly_calls":3,"gitaly_duration_s":0.013449,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014903,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001239,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002436,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.002587,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000279,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.00437,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000763,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000219,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1706,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20373940000000001,"duration_s":0.249308,"completed_at":"2026-06-29T21:59:52.144Z","load_balancing_strategy":"primary","db_duration_s":0.054857,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1990vm-test-run-nixbot-gitlab> gitlab # [ 324.313251] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.146Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","38f6765a1ccc42037bff68d9"],"class":"WebHooks::LogExecutionWorker","jid":"691f7b998facdbee437b1d1a","created_at":"2026-06-29T21:59:51.642Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-e2f7ac1819aa4cb8","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6c7b0794547656e351ecbc1831a0f0f96573ff82f0f692c378cde0395a6bfc09","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:51.649Z","job_size_bytes":2507,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-691f7b998facdbee437b1d1a: start","job_status":"start","queue_duration_s":0.497222,"scheduling_latency_s":0.497245}1991vm-test-run-nixbot-gitlab> gitlab # [ 324.331471] gitaly[1444]: time="2026-06-29T21:59:52.166Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 diskcache=d05d01f0-f7ab-4c75-92eb-47ae1f12da9f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:00:02.055" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:52.076" pid=1444 ref=refs/keep-around/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 remote_ip="2001:db8:1::1" revision=143c43e7a9f21ebec3698d7b35eda5ca9cef1512 span.kind=server system=grpc username=root1992vm-test-run-nixbot-gitlab> gitlab # [ 324.343981] gitaly[1444]: time="2026-06-29T21:59:52.167Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5020 command.oublock=8 command.real_time_ms=83 command.spawn_token_fork_ms=5 command.system_time_ms=34 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:00:02.055" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:59:52.076" grpc.time_ms=90.42 pid=1444 ref=refs/keep-around/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 remote_ip="2001:db8:1::1" revision=143c43e7a9f21ebec3698d7b35eda5ca9cef1512 span.kind=server system=grpc username=root1993vm-test-run-nixbot-gitlab> gitlab # [ 324.358979] gitaly[1444]: time="2026-06-29T21:59:52.174Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.174" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.174" grpc.time_ms=0.343 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994vm-test-run-nixbot-gitlab> gitlab # [ 324.372378] gitaly[1444]: time="2026-06-29T21:59:52.258Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=6 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:02.113" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.243" grpc.time_ms=14.855 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995vm-test-run-nixbot-gitlab> gitlab # [ 324.386178] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3491996vm-test-run-nixbot-gitlab> gitlab # [ 324.389303] nixbot[3944]: {"ts": "2026-06-29T21:59:52.274Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 \"HTTP/1.1 201 Created\""}1997vm-test-run-nixbot-gitlab> gitlab # [ 324.392225] nixbot[3944]: {"ts": "2026-06-29T21:59:52.282Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-143c43e7a9f2-b33760e5", "argv0": "bwrap"}1998vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512'1999vm-test-run-nixbot-gitlab> gitlab # [ 324.404250] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.318Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","38f6765a1ccc42037bff68d9"],"class":"WebHooks::LogExecutionWorker","jid":"691f7b998facdbee437b1d1a","created_at":"2026-06-29T21:59:51.642Z","trace_propagation_headers":{"sentry-trace":"24f22681628d416f90d8a4fad3a6784e-e2f7ac1819aa4cb8","baggage":"sentry-trace_id=24f22681628d416f90d8a4fad3a6784e,sentry-release=Unknown"},"correlation_id":"01KWAPAMCB2ZZVMAHQK1XA1DK8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6c7b0794547656e351ecbc1831a0f0f96573ff82f0f692c378cde0395a6bfc09","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:51.649Z","job_size_bytes":2507,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-691f7b998facdbee437b1d1a: done: 0.172146 sec","job_status":"done","queue_duration_s":0.497222,"scheduling_latency_s":0.497245,"redis_calls":10,"redis_duration_s":0.002249,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000321,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.043,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.043,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042868,"duration_s":0.172146,"completed_at":"2026-06-29T21:59:52.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000vm-test-run-nixbot-gitlab> gitlab # [ 324.448751] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.322Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b8e90f041d09a73f478cc1ce","created_at":"2026-06-29T21:59:52.005Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-ee2c27f01aab4ef0","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.012Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b8e90f041d09a73f478cc1ce: start","job_status":"start","queue_duration_s":0.309477,"scheduling_latency_s":0.309506}2001vm-test-run-nixbot-gitlab> gitlab # [ 324.466547] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2992002vm-test-run-nixbot-gitlab> gitlab # [ 324.469534] nixbot[3944]: {"ts": "2026-06-29T21:59:52.365Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b \"HTTP/1.1 201 Created\""}2003vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512', in 0.08 seconds)2004vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-29T21:59:51.910865Z', 'started_at': '2026-06-29T21:59:52.277110Z', 'finished_at': None}]2005vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.37 seconds)2006vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2007vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2008vm-test-run-nixbot-gitlab> gitlab # [ 324.472506] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.380Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b8e90f041d09a73f478cc1ce","created_at":"2026-06-29T21:59:52.005Z","trace_propagation_headers":{"sentry-trace":"a4821f556c644036ad49515dc7e866ca-ee2c27f01aab4ef0","baggage":"sentry-trace_id=a4821f556c644036ad49515dc7e866ca,sentry-release=Unknown"},"correlation_id":"01KWAPANGTHH6BZN2MYXVEV5A5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.012Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b8e90f041d09a73f478cc1ce: done: 0.0583 sec","job_status":"done","queue_duration_s":0.309477,"scheduling_latency_s":0.309506,"redis_calls":10,"redis_duration_s":0.006043,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000508,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052338,"duration_s":0.0583,"completed_at":"2026-06-29T21:59:52.380Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2009vm-test-run-nixbot-gitlab> gitlab # [ 324.517740] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.391Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"214777f190acc3334ead8c6a","created_at":"2026-06-29T21:59:52.176Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.180Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-214777f190acc3334ead8c6a: start","job_status":"start","queue_duration_s":0.210947,"scheduling_latency_s":0.210971}2010vm-test-run-nixbot-gitlab> gitlab # [ 324.534909] nixbot[3944]: INFO: ::ffff:127.0.0.1:50194 - "GET /api/repos/gitlab/root/test-flake/builds?commit=143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 200 OK2011vm-test-run-nixbot-gitlab> gitlab # [ 324.537483] gitaly[1444]: time="2026-06-29T21:59:52.447Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAQ9YPBQTXB8K272R1QJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.446" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.446" grpc.time_ms=0.576 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012vm-test-run-nixbot-gitlab> gitlab # [ 324.552063] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.437Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"214777f190acc3334ead8c6a","created_at":"2026-06-29T21:59:52.176Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.180Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-214777f190acc3334ead8c6a: done: 0.04593 sec","job_status":"done","queue_duration_s":0.210947,"scheduling_latency_s":0.210971,"redis_calls":7,"redis_duration_s":0.009389,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04593,"completed_at":"2026-06-29T21:59:52.437Z","load_balancing_strategy":"primary","db_duration_s":0.004757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2013vm-test-run-nixbot-gitlab> gitlab # [ 324.598199] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.443Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b1f75e76d7271f8ad513e2c3","created_at":"2026-06-29T21:59:52.183Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.188Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b1f75e76d7271f8ad513e2c3: start","job_status":"start","queue_duration_s":0.255373,"scheduling_latency_s":0.255429}2014vm-test-run-nixbot-gitlab> gitlab # [ 324.617677] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.522Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b1f75e76d7271f8ad513e2c3","created_at":"2026-06-29T21:59:52.183Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.188Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b1f75e76d7271f8ad513e2c3: done: 0.079351 sec","job_status":"done","queue_duration_s":0.255373,"scheduling_latency_s":0.255429,"redis_calls":10,"redis_duration_s":0.0075650000000000005,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.003493,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.079351,"completed_at":"2026-06-29T21:59:52.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015vm-test-run-nixbot-gitlab> gitlab # [ 324.663129] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.531Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f477d325edfacd0ec63b8352","created_at":"2026-06-29T21:59:52.255Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.258Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f477d325edfacd0ec63b8352: start","job_status":"start","queue_duration_s":0.272391,"scheduling_latency_s":0.272423}2016vm-test-run-nixbot-gitlab> gitlab # [ 324.680287] gitaly[1444]: time="2026-06-29T21:59:52.510Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=11 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAQ9YPBQTXB8K272R1QJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:02.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.491" grpc.time_ms=19.664 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2017vm-test-run-nixbot-gitlab> gitlab # [ 324.694429] nix-daemon[4269]: accepted connection from pid 4341, user nixbot2018vm-test-run-nixbot-gitlab> gitlab # [ 324.695446] nix-daemon[4269]: accepted connection from pid 4342, user nixbot2019vm-test-run-nixbot-gitlab> gitlab # [ 324.696503] nixbot[3944]: {"ts": "2026-06-29T21:59:52.574Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b \"HTTP/1.1 201 Created\""}2020vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.20 seconds)2021vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2022vm-test-run-nixbot-gitlab> gitlab # [ 324.699480] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/73944bd72e96e739664970d1897e8de7fe8fda3b HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1902023vm-test-run-nixbot-gitlab> gitlab # [ 324.702098] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 952024vm-test-run-nixbot-gitlab> gitlab # [ 324.704735] gitaly[1444]: time="2026-06-29T21:59:52.576Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAQE4ZN5GRVPR9V3ADV73 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.575" grpc.time_ms=0.702 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [ 324.717862] gitaly[1444]: time="2026-06-29T21:59:52.621Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:22.519" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.620" grpc.time_ms=1.106 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026vm-test-run-nixbot-gitlab> gitlab # [ 324.746719] gitaly[1444]: time="2026-06-29T21:59:52.661Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.560" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.660" grpc.time_ms=0.892 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027vm-test-run-nixbot-gitlab> gitlab # [ 324.792621] gitaly[1444]: time="2026-06-29T21:59:52.707Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPVK0KCXMQJ68DY6S5M3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:22.605" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:52.706" grpc.time_ms=1.14 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028vm-test-run-nixbot-gitlab> gitlab # [ 324.819166] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f477d325edfacd0ec63b8352","created_at":"2026-06-29T21:59:52.255Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-63591211da3e48bb","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.258Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f477d325edfacd0ec63b8352: done: 0.194021 sec","job_status":"done","queue_duration_s":0.272391,"scheduling_latency_s":0.272423,"gitaly_calls":3,"gitaly_duration_s":0.012452,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014669,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.0035,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00233,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000251,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002746,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000277,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00047,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.142819,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15892022,"duration_s":0.194021,"completed_at":"2026-06-29T21:59:52.725Z","load_balancing_strategy":"primary","db_duration_s":0.03297,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2029vm-test-run-nixbot-gitlab> gitlab # [ 324.891661] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:52.728Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8e266940d7ac35e1490aa2bc","created_at":"2026-06-29T21:59:52.343Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-3fb5dda87f5d4e63","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.346Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8e266940d7ac35e1490aa2bc: start","job_status":"start","queue_duration_s":0.381241,"scheduling_latency_s":0.381276}2030vm-test-run-nixbot-gitlab> gitlab # [ 325.012632] gitaly[1444]: time="2026-06-29T21:59:52.927Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:52.926" grpc.time_ms=1.062 pid=1444 span.kind=server system=grpc2031vm-test-run-nixbot-gitlab> gitlab # [ 325.072929] gitaly[1444]: time="2026-06-29T21:59:52.988Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:52.986" grpc.time_ms=0.98 pid=1444 span.kind=server system=grpc2032vm-test-run-nixbot-gitlab> gitlab # [ 325.126725] gitaly[1444]: time="2026-06-29T21:59:53.041Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:53.041" grpc.time_ms=0.807 pid=1444 span.kind=server system=grpc2033vm-test-run-nixbot-gitlab> gitlab # [ 325.148877] gitaly[1444]: time="2026-06-29T21:59:53.056Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ diskcache=94c2f439-fcb0-480b-a13a-06a1f33567a9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:00:22.764" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.865" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034vm-test-run-nixbot-gitlab> gitlab # [ 325.167823] gitaly[1444]: time="2026-06-29T21:59:53.057Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=212 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7439 command.oublock=0 command.real_time_ms=182 command.spawn_token_fork_ms=1 command.system_time_ms=110 command.user_time_ms=102 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:00:22.764" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:52.865" grpc.time_ms=190.872 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2035vm-test-run-nixbot-gitlab> gitlab # [ 325.291991] gitaly[1444]: time="2026-06-29T21:59:53.207Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:23.104" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:53.205" grpc.time_ms=1.419 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2036vm-test-run-nixbot-gitlab> gitlab # [ 325.362490] gitaly[1444]: time="2026-06-29T21:59:53.277Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:23.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:53.275" grpc.time_ms=1.526 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037vm-test-run-nixbot-gitlab> gitlab # [ 325.459041] gitaly[1444]: time="2026-06-29T21:59:53.374Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAPZX65FMTMPBZCF1EXSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:23.271" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:53.372" grpc.time_ms=1.466 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2038vm-test-run-nixbot-gitlab> gitlab # [ 325.488869] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.393Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8e266940d7ac35e1490aa2bc","created_at":"2026-06-29T21:59:52.343Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-3fb5dda87f5d4e63","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.346Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2518,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8e266940d7ac35e1490aa2bc: rescheduled"}2039vm-test-run-nixbot-gitlab> gitlab # [ 325.513608] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.411Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8e266940d7ac35e1490aa2bc","created_at":"2026-06-29T21:59:52.343Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-3fb5dda87f5d4e63","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.346Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8e266940d7ac35e1490aa2bc: done: 0.683179 sec","job_status":"done","queue_duration_s":0.381241,"scheduling_latency_s":0.381276,"gitaly_calls":4,"gitaly_duration_s":0.21082,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.035036,"redis_read_bytes":185,"redis_write_bytes":23554,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001734,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003094,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.008936,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.005802,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008944,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00049,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000793,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003509,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.027,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.031,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.375059,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.62507802,"duration_s":0.683179,"completed_at":"2026-06-29T21:59:53.411Z","load_balancing_strategy":"primary","db_duration_s":0.095453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2040vm-test-run-nixbot-gitlab> gitlab # [ 325.573706] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bc7350e08eb4301584dd33dd","created_at":"2026-06-29T21:59:52.605Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-5204dfbb2b4a452d","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.608Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc7350e08eb4301584dd33dd: start","job_status":"start","queue_duration_s":0.806231,"scheduling_latency_s":0.806277}2041vm-test-run-nixbot-gitlab> gitlab # [ 325.590674] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.480Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bc7350e08eb4301584dd33dd","created_at":"2026-06-29T21:59:52.605Z","trace_propagation_headers":{"sentry-trace":"01ef62b95f934a8584464e0ba31c5531-5204dfbb2b4a452d","baggage":"sentry-trace_id=01ef62b95f934a8584464e0ba31c5531,sentry-release=Unknown"},"correlation_id":"01KWAPAPVK0KCXMQJ68DY6S5M3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.608Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc7350e08eb4301584dd33dd: done: 0.065281 sec","job_status":"done","queue_duration_s":0.806231,"scheduling_latency_s":0.806277,"redis_calls":7,"redis_duration_s":0.003022,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065281,"completed_at":"2026-06-29T21:59:53.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007425,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042vm-test-run-nixbot-gitlab> gitlab # [ 325.633543] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8e629d45f00ba2d02c247a1d","created_at":"2026-06-29T21:59:52.842Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.850Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8e629d45f00ba2d02c247a1d: start","job_status":"start","queue_duration_s":0.632818,"scheduling_latency_s":0.632855}2043vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2044vm-test-run-nixbot-gitlab> gitlab # [ 325.740337] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.654Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8e629d45f00ba2d02c247a1d","created_at":"2026-06-29T21:59:52.842Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:52.850Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8e629d45f00ba2d02c247a1d: done: 0.171938 sec","job_status":"done","queue_duration_s":0.632818,"scheduling_latency_s":0.632855,"redis_calls":15,"redis_duration_s":0.005273,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000668,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.00049,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0025,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000856,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.138576,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047286,"duration_s":0.171938,"completed_at":"2026-06-29T21:59:53.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2045vm-test-run-nixbot-gitlab> gitlab # [ 325.831316] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.657Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"132f1866c9cfb463eac3a102","created_at":"2026-06-29T21:59:53.066Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.078Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-132f1866c9cfb463eac3a102: start","job_status":"start","queue_duration_s":0.579369,"scheduling_latency_s":0.579394}2046vm-test-run-nixbot-gitlab> gitlab # [ 325.850320] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.700Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"132f1866c9cfb463eac3a102","created_at":"2026-06-29T21:59:53.066Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.078Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-132f1866c9cfb463eac3a102: done: 0.042821 sec","job_status":"done","queue_duration_s":0.579369,"scheduling_latency_s":0.579394,"redis_calls":10,"redis_duration_s":0.003596,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049538,"duration_s":0.042821,"completed_at":"2026-06-29T21:59:53.700Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047vm-test-run-nixbot-gitlab> gitlab # [ 325.897754] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9b362dadbf78b8bbce03241f","created_at":"2026-06-29T21:59:53.106Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.117Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9b362dadbf78b8bbce03241f: start","job_status":"start","queue_duration_s":0.584467,"scheduling_latency_s":0.58449}2048vm-test-run-nixbot-gitlab> gitlab # [ 325.914073] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9b362dadbf78b8bbce03241f","created_at":"2026-06-29T21:59:53.106Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.117Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9b362dadbf78b8bbce03241f: done: 0.053177 sec","job_status":"done","queue_duration_s":0.584467,"scheduling_latency_s":0.58449,"redis_calls":10,"redis_duration_s":0.0017820000000000002,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034888,"duration_s":0.053177,"completed_at":"2026-06-29T21:59:53.754Z","load_balancing_strategy":"primary","db_duration_s":0.018916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.26 seconds)2050vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2051vm-test-run-nixbot-gitlab> gitlab # [ 325.958968] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.756Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"08f36d4ac3541634c4b4ce09","created_at":"2026-06-29T21:59:53.641Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-5944f190230e4f4a","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:67b590dcd91ac167074499d0a8fd80c9e881a69188b9aac5de151f7ccefc952e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.644Z","job_size_bytes":464,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-08f36d4ac3541634c4b4ce09: start","job_status":"start","queue_duration_s":0.111847,"scheduling_latency_s":0.11187}2052vm-test-run-nixbot-gitlab> gitlab # [ 325.975266] sidekiq-cluster[2518]: {"severity":"WARN","time":"2026-06-29T21:59:53.811Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"08f36d4ac3541634c4b4ce09","created_at":"2026-06-29T21:59:53.641Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-5944f190230e4f4a","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:67b590dcd91ac167074499d0a8fd80c9e881a69188b9aac5de151f7ccefc952e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.644Z","job_size_bytes":464,"pid":2518,"sidekiq_tid":"m4ba","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-08f36d4ac3541634c4b4ce09: fail: 0.05382 sec","job_status":"fail","queue_duration_s":0.111847,"scheduling_latency_s":0.11187,"redis_calls":7,"redis_duration_s":0.001862,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001303,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05382,"completed_at":"2026-06-29T21:59:53.810Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.013992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053vm-test-run-nixbot-gitlab> gitlab # [ 326.358560] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.815Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5fcc120fef8be209bd56a58c","created_at":"2026-06-29T21:59:53.145Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.155Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5fcc120fef8be209bd56a58c: start","job_status":"start","queue_duration_s":0.659556,"scheduling_latency_s":0.659581}2054vm-test-run-nixbot-gitlab> gitlab # [ 326.374994] gitaly[1444]: time="2026-06-29T21:59:53.824Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPARK3Z4G9X1EPC842CVQ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:23.022" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:53.823" grpc.time_ms=1.118 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055vm-test-run-nixbot-gitlab> gitlab # [ 326.388593] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1672056vm-test-run-nixbot-gitlab> gitlab # [ 326.391408] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5fcc120fef8be209bd56a58c","created_at":"2026-06-29T21:59:53.145Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.155Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5fcc120fef8be209bd56a58c: done: 0.062951 sec","job_status":"done","queue_duration_s":0.659556,"scheduling_latency_s":0.659581,"redis_calls":13,"redis_duration_s":0.009273,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000719,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001434,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003646,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061,"duration_s":0.062951,"completed_at":"2026-06-29T21:59:53.878Z","load_balancing_strategy":"primary","db_duration_s":0.01238,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057vm-test-run-nixbot-gitlab> gitlab # [ 326.438337] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.880Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dd5cb18a106694b8ba9d901c","created_at":"2026-06-29T21:59:53.157Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.166Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dd5cb18a106694b8ba9d901c: start","job_status":"start","queue_duration_s":0.713417,"scheduling_latency_s":0.713439}2058vm-test-run-nixbot-gitlab> gitlab # [ 326.455502] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dd5cb18a106694b8ba9d901c","created_at":"2026-06-29T21:59:53.157Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.166Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dd5cb18a106694b8ba9d901c: done: 0.037659 sec","job_status":"done","queue_duration_s":0.713417,"scheduling_latency_s":0.713439,"redis_calls":10,"redis_duration_s":0.003192,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004071,"duration_s":0.037659,"completed_at":"2026-06-29T21:59:53.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059vm-test-run-nixbot-gitlab> gitlab # [ 326.502624] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.919Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2c4ebac9b2898fc0c989fde4","created_at":"2026-06-29T21:59:53.168Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.176Z","job_size_bytes":19,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2c4ebac9b2898fc0c989fde4: start","job_status":"start","queue_duration_s":0.743167,"scheduling_latency_s":0.743187}2060vm-test-run-nixbot-gitlab> gitlab # [ 326.519387] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.955Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2c4ebac9b2898fc0c989fde4","created_at":"2026-06-29T21:59:53.168Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.176Z","job_size_bytes":19,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2c4ebac9b2898fc0c989fde4: done: 0.035961 sec","job_status":"done","queue_duration_s":0.743167,"scheduling_latency_s":0.743187,"redis_calls":11,"redis_duration_s":0.003356,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000672,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002151,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000533,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035864,"duration_s":0.035961,"completed_at":"2026-06-29T21:59:53.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061vm-test-run-nixbot-gitlab> gitlab # [ 326.565277] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.957Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f0f64f21e6783f3ec32eaf03","created_at":"2026-06-29T21:59:53.182Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.189Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f0f64f21e6783f3ec32eaf03: start","job_status":"start","queue_duration_s":0.767757,"scheduling_latency_s":0.767776}2062vm-test-run-nixbot-gitlab> gitlab # [ 326.583514] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f0f64f21e6783f3ec32eaf03","created_at":"2026-06-29T21:59:53.182Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.189Z","job_size_bytes":65,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f0f64f21e6783f3ec32eaf03: done: 0.033193 sec","job_status":"done","queue_duration_s":0.767757,"scheduling_latency_s":0.767776,"redis_calls":10,"redis_duration_s":0.002856,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000599,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001636,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038842,"duration_s":0.033193,"completed_at":"2026-06-29T21:59:53.990Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063vm-test-run-nixbot-gitlab> gitlab # [ 326.630593] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:53.992Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ad11e79f6ca735926b3adb24","created_at":"2026-06-29T21:59:53.191Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.194Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ad11e79f6ca735926b3adb24: start","job_status":"start","queue_duration_s":0.797679,"scheduling_latency_s":0.797697}2064vm-test-run-nixbot-gitlab> gitlab # [ 326.647724] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.025Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ad11e79f6ca735926b3adb24","created_at":"2026-06-29T21:59:53.191Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.194Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ad11e79f6ca735926b3adb24: done: 0.032775 sec","job_status":"done","queue_duration_s":0.797679,"scheduling_latency_s":0.797697,"redis_calls":7,"redis_duration_s":0.002471,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032775,"completed_at":"2026-06-29T21:59:54.024Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065vm-test-run-nixbot-gitlab> gitlab # [ 326.692419] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.026Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f8f82da7221c05711bf227dc","created_at":"2026-06-29T21:59:53.280Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.293Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f8f82da7221c05711bf227dc: start","job_status":"start","queue_duration_s":0.733586,"scheduling_latency_s":0.733607}2066vm-test-run-nixbot-gitlab> gitlab # [ 326.708839] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.066Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f8f82da7221c05711bf227dc","created_at":"2026-06-29T21:59:53.280Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.293Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f8f82da7221c05711bf227dc: done: 0.040347 sec","job_status":"done","queue_duration_s":0.733586,"scheduling_latency_s":0.733607,"redis_calls":10,"redis_duration_s":0.0029779999999999997,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048236,"duration_s":0.040347,"completed_at":"2026-06-29T21:59:54.066Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2067vm-test-run-nixbot-gitlab> gitlab # [ 326.753140] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.068Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9133a5075edb86842c1d1c46","created_at":"2026-06-29T21:59:53.294Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.305Z","job_size_bytes":6,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9133a5075edb86842c1d1c46: start","job_status":"start","queue_duration_s":0.762892,"scheduling_latency_s":0.762911}2068vm-test-run-nixbot-gitlab> gitlab # [ 326.768879] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.093Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9133a5075edb86842c1d1c46","created_at":"2026-06-29T21:59:53.294Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.305Z","job_size_bytes":6,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9133a5075edb86842c1d1c46: done: 0.024888 sec","job_status":"done","queue_duration_s":0.762892,"scheduling_latency_s":0.762911,"redis_calls":10,"redis_duration_s":0.002857,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00056,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039972,"duration_s":0.024888,"completed_at":"2026-06-29T21:59:54.093Z","load_balancing_strategy":"primary","db_duration_s":0.0038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069vm-test-run-nixbot-gitlab> gitlab # [ 326.812506] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.095Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"14871fb3ec3daca68c850b05","created_at":"2026-06-29T21:59:53.394Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.399Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-14871fb3ec3daca68c850b05: start","job_status":"start","queue_duration_s":0.695329,"scheduling_latency_s":0.695347}2070vm-test-run-nixbot-gitlab> gitlab # [ 326.828133] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:54.115Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"14871fb3ec3daca68c850b05","created_at":"2026-06-29T21:59:53.394Z","trace_propagation_headers":{"sentry-trace":"4054f1c9cf024f51978d79918179e1e0-2a2ae4ddc80a496c","baggage":"sentry-trace_id=4054f1c9cf024f51978d79918179e1e0,sentry-release=Unknown"},"correlation_id":"01KWAPAPZX65FMTMPBZCF1EXSQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:53.399Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-14871fb3ec3daca68c850b05: done: 0.020365 sec","job_status":"done","queue_duration_s":0.695329,"scheduling_latency_s":0.695347,"redis_calls":8,"redis_duration_s":0.002461,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001929,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020365,"completed_at":"2026-06-29T21:59:54.115Z","load_balancing_strategy":"primary","db_duration_s":0.003112,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2071vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2072vm-test-run-nixbot-gitlab> gitlab # [ 327.151797] gitaly[1444]: time="2026-06-29T21:59:55.066Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPASTND97K47BFT2RYEK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:25.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:55.065" grpc.time_ms=1.387 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.24 seconds)2074vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2075vm-test-run-nixbot-gitlab> gitlab # [ 327.201530] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1502076vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2077vm-test-run-nixbot-gitlab> gitlab # [ 328.353187] gitaly[1444]: time="2026-06-29T21:59:56.267Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAV0E5NJ5ZKKEZ6VVZZ8J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:26.264" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:56.265" grpc.time_ms=1.362 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.20 seconds)2079vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2080vm-test-run-nixbot-gitlab> gitlab # [ 328.401179] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1402081vm-test-run-nixbot-gitlab> gitlab # [ 328.985825] gitaly[1444]: time="2026-06-29T21:59:56.900Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:26.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:56.898" grpc.time_ms=1.678 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082vm-test-run-nixbot-gitlab> gitlab # [ 329.078586] gitaly[1444]: time="2026-06-29T21:59:56.993Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:06.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:56.973" grpc.time_ms=20.326 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2083vm-test-run-nixbot-gitlab> gitlab # [ 329.158559] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e722cc448f0e3ce9c03d3858","created_at":"2026-06-29T21:59:57.061Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-1e1b1cde47e44e8d","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.068Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e722cc448f0e3ce9c03d3858: start","job_status":"start","queue_duration_s":0.002862,"scheduling_latency_s":0.002927}2084vm-test-run-nixbot-gitlab> gitlab # [ 329.207014] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3072085vm-test-run-nixbot-gitlab> gitlab # [ 329.214672] nixbot[3944]: {"ts": "2026-06-29T21:59:57.096Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 \"HTTP/1.1 201 Created\""}2086vm-test-run-nixbot-gitlab> gitlab # [ 329.261091] gitaly[1444]: time="2026-06-29T21:59:57.176Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVXEZEZD4ZRJD1TVT7AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:27.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.174" grpc.time_ms=1.488 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087vm-test-run-nixbot-gitlab> gitlab # [ 329.330414] gitaly[1444]: time="2026-06-29T21:59:57.245Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVXEZEZD4ZRJD1TVT7AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:07.205" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:57.226" grpc.time_ms=19.187 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088vm-test-run-nixbot-gitlab> gitlab # [ 329.379024] gitaly[1444]: time="2026-06-29T21:59:57.292Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:57.291" grpc.time_ms=0.864 pid=1444 span.kind=server system=grpc2089vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2090vm-test-run-nixbot-gitlab> gitlab # [ 329.418140] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2292091vm-test-run-nixbot-gitlab> gitlab # [ 329.424926] nixbot[3944]: {"ts": "2026-06-29T21:59:57.334Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 \"HTTP/1.1 201 Created\""}2092vm-test-run-nixbot-gitlab> gitlab # [ 329.436960] gitaly[1444]: time="2026-06-29T21:59:57.350Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:57.350" grpc.time_ms=0.75 pid=1444 span.kind=server system=grpc2093vm-test-run-nixbot-gitlab> gitlab # [ 329.480097] gitaly[1444]: time="2026-06-29T21:59:57.393Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:57.392" grpc.time_ms=0.435 pid=1444 span.kind=server system=grpc2094vm-test-run-nixbot-gitlab> gitlab # [ 329.502997] gitaly[1444]: time="2026-06-29T21:59:57.418Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 diskcache=0fdd254b-5fdd-487c-925f-a7540aa60a87 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:00:27.128" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:57.229" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095vm-test-run-nixbot-gitlab> gitlab # [ 329.513764] gitaly[1444]: time="2026-06-29T21:59:57.418Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=142 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7180 command.oublock=0 command.real_time_ms=182 command.spawn_token_fork_ms=1 command.system_time_ms=68 command.user_time_ms=74 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:00:27.128" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:57.229" grpc.time_ms=189.251 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096vm-test-run-nixbot-gitlab> gitlab # [ 329.528834] gitaly[1444]: time="2026-06-29T21:59:57.429Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAW5CWABDZ9RK3DB4DMBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:27.424" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.424" grpc.time_ms=4.165 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097vm-test-run-nixbot-gitlab> gitlab # [ 329.550983] gitaly[1444]: time="2026-06-29T21:59:57.443Z" level=info msg="finished unary call with code OK" catfile.duration_ms=8 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=8 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAW65MKV6Q4NG9PQZ01CN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:27.334" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.434" grpc.time_ms=8.615 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098vm-test-run-nixbot-gitlab> gitlab # [ 329.571142] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1042099vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.17 seconds)2100vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': '2026-06-29T21:59:57.041Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T21:59:57.290Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2101vm-test-run-nixbot-gitlab> gitlab # [ 329.587370] gitaly[1444]: time="2026-06-29T21:59:57.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAW5CWABDZ9RK3DB4DMBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:00:07.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:57.491" grpc.time_ms=10.837 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102vm-test-run-nixbot-gitlab> gitlab # [ 329.642926] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2012103vm-test-run-nixbot-gitlab> gitlab # [ 329.646901] nixbot[3944]: {"ts": "2026-06-29T21:59:57.560Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/143c43e7a9f21ebec3698d7b35eda5ca9cef1512 \"HTTP/1.1 201 Created\""}2104vm-test-run-nixbot-gitlab> gitlab # [ 329.664135] gitaly[1444]: time="2026-06-29T21:59:57.579Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:00:27.477" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.577" grpc.time_ms=1.268 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105vm-test-run-nixbot-gitlab> gitlab # [ 329.698857] gitaly[1444]: time="2026-06-29T21:59:57.614Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:27.512" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.612" grpc.time_ms=0.967 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106vm-test-run-nixbot-gitlab> gitlab # [ 329.757003] gitaly[1444]: time="2026-06-29T21:59:57.671Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:00:27.570" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:59:57.670" grpc.time_ms=0.781 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107vm-test-run-nixbot-gitlab> gitlab # [ 329.773305] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.684Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c6651275e44c90d995b36aea","created_at":"2026-06-29T21:59:57.682Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"e722cc448f0e3ce9c03d3858","job_size_bytes":3,"pid":2518,"job_status":"deduplicated","message":"PipelineProcessWorker JID-c6651275e44c90d995b36aea: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2108vm-test-run-nixbot-gitlab> gitlab # [ 329.791128] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.686Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e722cc448f0e3ce9c03d3858","created_at":"2026-06-29T21:59:57.061Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-1e1b1cde47e44e8d","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.068Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2518,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e722cc448f0e3ce9c03d3858: rescheduled"}2109vm-test-run-nixbot-gitlab> gitlab # [ 329.807550] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.698Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e722cc448f0e3ce9c03d3858","created_at":"2026-06-29T21:59:57.061Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-1e1b1cde47e44e8d","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.068Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e722cc448f0e3ce9c03d3858: done: 0.627105 sec","job_status":"done","queue_duration_s":0.002862,"scheduling_latency_s":0.002927,"gitaly_calls":4,"gitaly_duration_s":0.30212,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.037094,"redis_read_bytes":262,"redis_write_bytes":23386,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002072,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00251,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001032,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.012261,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14467,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.010038,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000474,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001496,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.005139,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.252195,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.56669686,"duration_s":0.627105,"completed_at":"2026-06-29T21:59:57.698Z","load_balancing_strategy":"primary","db_duration_s":0.053622,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2110vm-test-run-nixbot-gitlab> gitlab # [ 329.867244] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c49154500f13468c1eb9c9e2","created_at":"2026-06-29T21:59:57.215Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.219Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c49154500f13468c1eb9c9e2: start","job_status":"start","queue_duration_s":0.486005,"scheduling_latency_s":0.486032}2111vm-test-run-nixbot-gitlab> gitlab # [ 329.884698] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c49154500f13468c1eb9c9e2","created_at":"2026-06-29T21:59:57.215Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.219Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c49154500f13468c1eb9c9e2: done: 0.08128 sec","job_status":"done","queue_duration_s":0.486005,"scheduling_latency_s":0.486032,"redis_calls":7,"redis_duration_s":0.020572,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005735,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08128,"completed_at":"2026-06-29T21:59:57.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025912,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2112vm-test-run-nixbot-gitlab> gitlab # [ 329.929057] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.789Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bedb36a6b3346a8dce10dcaf","created_at":"2026-06-29T21:59:57.526Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.529Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bedb36a6b3346a8dce10dcaf: start","job_status":"start","queue_duration_s":0.259824,"scheduling_latency_s":0.259847}2113vm-test-run-nixbot-gitlab> gitlab # [ 329.945876] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 982114vm-test-run-nixbot-gitlab> gitlab # [ 329.948138] gitlab-workhorse[1369]: time="2026-06-29T21:59:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672115vm-test-run-nixbot-gitlab> gitlab # [ 329.950098] gitlab-workhorse[1369]: gitlab 2001:db8:1::1 - - [2026/06/29:21:59:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1002116vm-test-run-nixbot-gitlab> gitlab # [ 329.952186] gitaly[1444]: time="2026-06-29T21:59:57.723Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPAWDSKQ1PM2TQXKWGVV7R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:57.711" grpc.time_ms=12.34 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc2117vm-test-run-nixbot-gitlab> gitlab # [ 329.967152] gitaly[1444]: time="2026-06-29T21:59:57.826Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAWGZHTS11S9XZCQKPK3S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:57.802" pid=1444 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2118vm-test-run-nixbot-gitlab> gitlab # [ 329.977819] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.830Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bedb36a6b3346a8dce10dcaf","created_at":"2026-06-29T21:59:57.526Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.529Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bedb36a6b3346a8dce10dcaf: done: 0.041219 sec","job_status":"done","queue_duration_s":0.259824,"scheduling_latency_s":0.259847,"redis_calls":7,"redis_duration_s":0.004373,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041219,"completed_at":"2026-06-29T21:59:57.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119vm-test-run-nixbot-gitlab> gitlab # [ 330.022557] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"18b5dc1100f29cca099545d3","created_at":"2026-06-29T21:59:57.540Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.543Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-18b5dc1100f29cca099545d3: start","job_status":"start","queue_duration_s":0.292606,"scheduling_latency_s":0.292628}2120vm-test-run-nixbot-gitlab> gitlab # [ 330.039505] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"18b5dc1100f29cca099545d3","created_at":"2026-06-29T21:59:57.540Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.543Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-18b5dc1100f29cca099545d3: done: 0.028776 sec","job_status":"done","queue_duration_s":0.292606,"scheduling_latency_s":0.292628,"redis_calls":7,"redis_duration_s":0.00118,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028776,"completed_at":"2026-06-29T21:59:57.864Z","load_balancing_strategy":"primary","db_duration_s":0.005878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121vm-test-run-nixbot-gitlab> gitlab # [ 330.082634] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0cf39e65078fa3ffd3bce103","created_at":"2026-06-29T21:59:57.553Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.556Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0cf39e65078fa3ffd3bce103: start","job_status":"start","queue_duration_s":0.310172,"scheduling_latency_s":0.310192}2122vm-test-run-nixbot-gitlab> gitlab # [ 330.100377] nix-daemon[4269]: accepted connection from pid 4455, user nixbot2123vm-test-run-nixbot-gitlab> gitlab # [ 330.101476] gitaly[1444]: time="2026-06-29T21:59:57.826Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAWGZHTS11S9XZCQKPK3S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:57.802" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc2124vm-test-run-nixbot-gitlab> gitlab # [ 330.115787] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0cf39e65078fa3ffd3bce103","created_at":"2026-06-29T21:59:57.553Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.556Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0cf39e65078fa3ffd3bce103: done: 0.027668 sec","job_status":"done","queue_duration_s":0.310172,"scheduling_latency_s":0.310192,"redis_calls":10,"redis_duration_s":0.001726,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000416,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027668,"completed_at":"2026-06-29T21:59:57.893Z","load_balancing_strategy":"primary","db_duration_s":0.003386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125vm-test-run-nixbot-gitlab> gitlab # [ 330.162660] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.895Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1792d808bbd0904a3559a4f6","created_at":"2026-06-29T21:59:57.556Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.558Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1792d808bbd0904a3559a4f6: start","job_status":"start","queue_duration_s":0.336412,"scheduling_latency_s":0.336433}2126vm-test-run-nixbot-gitlab> gitlab # [ 330.182064] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1792d808bbd0904a3559a4f6","created_at":"2026-06-29T21:59:57.556Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.558Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1792d808bbd0904a3559a4f6: done: 0.022337 sec","job_status":"done","queue_duration_s":0.336412,"scheduling_latency_s":0.336433,"redis_calls":7,"redis_duration_s":0.001077,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022337,"completed_at":"2026-06-29T21:59:57.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127vm-test-run-nixbot-gitlab> gitlab # [ 330.228270] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"87255fc3aa77c40c5c6e6a80","created_at":"2026-06-29T21:59:57.559Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.562Z","job_size_bytes":19,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-87255fc3aa77c40c5c6e6a80: start","job_status":"start","queue_duration_s":0.356131,"scheduling_latency_s":0.35615}2128vm-test-run-nixbot-gitlab> gitlab # [ 330.245011] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"87255fc3aa77c40c5c6e6a80","created_at":"2026-06-29T21:59:57.559Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.562Z","job_size_bytes":19,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-87255fc3aa77c40c5c6e6a80: done: 0.025183 sec","job_status":"done","queue_duration_s":0.356131,"scheduling_latency_s":0.35615,"redis_calls":8,"redis_duration_s":0.0013109999999999999,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000345,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025183,"completed_at":"2026-06-29T21:59:57.943Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129vm-test-run-nixbot-gitlab> gitlab # [ 330.289079] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.945Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"061f9414742e0be909eb6d21","created_at":"2026-06-29T21:59:57.564Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.567Z","job_size_bytes":66,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-061f9414742e0be909eb6d21: start","job_status":"start","queue_duration_s":0.37799,"scheduling_latency_s":0.378006}2130vm-test-run-nixbot-gitlab> gitlab # [ 330.307649] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.963Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"061f9414742e0be909eb6d21","created_at":"2026-06-29T21:59:57.564Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.567Z","job_size_bytes":66,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-061f9414742e0be909eb6d21: done: 0.018004 sec","job_status":"done","queue_duration_s":0.37799,"scheduling_latency_s":0.378006,"redis_calls":7,"redis_duration_s":0.0011220000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018004,"completed_at":"2026-06-29T21:59:57.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131vm-test-run-nixbot-gitlab> gitlab # [ 330.355168] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.964Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8b2d6ad5762d516e291825e3","created_at":"2026-06-29T21:59:57.568Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.570Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8b2d6ad5762d516e291825e3: start","job_status":"start","queue_duration_s":0.394116,"scheduling_latency_s":0.394137}2132vm-test-run-nixbot-gitlab> gitlab # [ 330.373099] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.993Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8b2d6ad5762d516e291825e3","created_at":"2026-06-29T21:59:57.568Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.570Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8b2d6ad5762d516e291825e3: done: 0.028701 sec","job_status":"done","queue_duration_s":0.394116,"scheduling_latency_s":0.394137,"redis_calls":7,"redis_duration_s":0.001192,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028701,"completed_at":"2026-06-29T21:59:57.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002834,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133vm-test-run-nixbot-gitlab> gitlab # [ 330.418989] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:57.994Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"df99054f78d264e6e4993f49","created_at":"2026-06-29T21:59:57.629Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.631Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-df99054f78d264e6e4993f49: start","job_status":"start","queue_duration_s":0.362652,"scheduling_latency_s":0.362671}2134vm-test-run-nixbot-gitlab> gitlab # [ 330.436151] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.015Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"df99054f78d264e6e4993f49","created_at":"2026-06-29T21:59:57.629Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.631Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-df99054f78d264e6e4993f49: done: 0.020979 sec","job_status":"done","queue_duration_s":0.362652,"scheduling_latency_s":0.362671,"redis_calls":7,"redis_duration_s":0.001313,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020979,"completed_at":"2026-06-29T21:59:58.015Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135vm-test-run-nixbot-gitlab> gitlab # [ 330.481059] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.016Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f08d78220529663042f33e5","created_at":"2026-06-29T21:59:57.635Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.638Z","job_size_bytes":6,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f08d78220529663042f33e5: start","job_status":"start","queue_duration_s":0.378462,"scheduling_latency_s":0.37848}2136vm-test-run-nixbot-gitlab> gitlab # [ 330.498664] gitaly[1444]: time="2026-06-29T21:59:57.826Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAWGZHTS11S9XZCQKPK3S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:59:57.802" grpc.time_ms=24.373 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc2137vm-test-run-nixbot-gitlab> gitlab # [ 330.513618] nix-daemon[4269]: accepted connection from pid 4475, user nixbot2138vm-test-run-nixbot-gitlab> gitlab # [ 330.514691] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f08d78220529663042f33e5","created_at":"2026-06-29T21:59:57.635Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.638Z","job_size_bytes":6,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f08d78220529663042f33e5: done: 0.016502 sec","job_status":"done","queue_duration_s":0.378462,"scheduling_latency_s":0.37848,"redis_calls":7,"redis_duration_s":0.0012799999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016502,"completed_at":"2026-06-29T21:59:58.033Z","load_balancing_strategy":"primary","db_duration_s":0.00032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139vm-test-run-nixbot-gitlab> gitlab # [ 330.559854] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"7a84e6ade0ae6e19b1420a65","created_at":"2026-06-29T21:59:57.639Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.643Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7a84e6ade0ae6e19b1420a65: start","job_status":"start","queue_duration_s":0.39076,"scheduling_latency_s":0.390779}2140vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses2141vm-test-run-nixbot-gitlab> gitlab # [ 330.576447] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"7a84e6ade0ae6e19b1420a65","created_at":"2026-06-29T21:59:57.639Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.643Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7a84e6ade0ae6e19b1420a65: done: 0.042572 sec","job_status":"done","queue_duration_s":0.39076,"scheduling_latency_s":0.390779,"redis_calls":10,"redis_duration_s":0.002296,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000323,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007002,"duration_s":0.042572,"completed_at":"2026-06-29T21:59:58.077Z","load_balancing_strategy":"primary","db_duration_s":0.005309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142vm-test-run-nixbot-gitlab> gitlab # [ 330.619151] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.079Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"f5b393e9d202c1fcd56bc73b","created_at":"2026-06-29T21:59:57.686Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.690Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5b393e9d202c1fcd56bc73b: start","job_status":"start","queue_duration_s":0.388905,"scheduling_latency_s":0.388936}2143vm-test-run-nixbot-gitlab> gitlab # [ 330.635186] gitaly[1444]: time="2026-06-29T21:59:58.248Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:58.247" grpc.time_ms=0.796 pid=1444 span.kind=server system=grpc2144vm-test-run-nixbot-gitlab> gitlab # [ 330.645075] gitaly[1444]: time="2026-06-29T21:59:58.299Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:58.298" grpc.time_ms=0.832 pid=1444 span.kind=server system=grpc2145vm-test-run-nixbot-gitlab> gitlab # [ 330.654997] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.542Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"f5b393e9d202c1fcd56bc73b","created_at":"2026-06-29T21:59:57.686Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-2fa9fc91ccb5445b","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:57.690Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5b393e9d202c1fcd56bc73b: done: 0.463114 sec","job_status":"done","queue_duration_s":0.388905,"scheduling_latency_s":0.388936,"gitaly_calls":3,"gitaly_duration_s":0.186714,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.038833,"redis_read_bytes":144,"redis_write_bytes":16875,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00093,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002116,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.004949,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11224,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.016852,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000501,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.005851,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.006704,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":26,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":26,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200112,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.40909544,"duration_s":0.463114,"completed_at":"2026-06-29T21:59:58.542Z","load_balancing_strategy":"primary","db_duration_s":0.051787,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2146vm-test-run-nixbot-gitlab> gitlab # [ 330.707631] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.544Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f4b40c21349d378bd79fc2a2","created_at":"2026-06-29T21:59:58.175Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-c2e81bcdca0d44cf","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:58.178Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f4b40c21349d378bd79fc2a2: start","job_status":"start","queue_duration_s":0.365825,"scheduling_latency_s":0.365853}2147vm-test-run-nixbot-gitlab> gitlab # [ 330.724372] gitaly[1444]: time="2026-06-29T21:59:58.347Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:59:58.346" grpc.time_ms=0.656 pid=1444 span.kind=server system=grpc2148vm-test-run-nixbot-gitlab> gitlab # [ 330.734350] gitaly[1444]: time="2026-06-29T21:59:58.355Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPAVKK6ENW6Q8TK9K5N1E7 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:00:28.087" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:59:58.187" operations="[repack_refs]" pid=1444 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149vm-test-run-nixbot-gitlab> gitlab # [ 330.744565] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.612Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f4b40c21349d378bd79fc2a2","created_at":"2026-06-29T21:59:58.175Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-c2e81bcdca0d44cf","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:58.178Z","job_size_bytes":23,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f4b40c21349d378bd79fc2a2: done: 0.067562 sec","job_status":"done","queue_duration_s":0.365825,"scheduling_latency_s":0.365853,"redis_calls":7,"redis_duration_s":0.002447,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067562,"completed_at":"2026-06-29T21:59:58.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018912,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2150vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/143c43e7a9f21ebec3698d7b35eda5ca9cef1512/statuses, in 0.23 seconds)2151vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:59:52.247Z', 'started_at': None, 'finished_at': '2026-06-29T21:59:57.041Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '143c43e7a9f21ebec3698d7b35eda5ca9cef1512', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-06-29T21:59:57.290Z', 'started_at': None, 'finished_at': '2026-06-29T21:59:57.529Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2152vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 6.31 seconds)2153vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 396.98 seconds)2154vm-test-run-nixbot-gitlab> gitlab # [ 330.790578] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.614Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e174cd320bb9461717a56b51","created_at":"2026-06-29T21:59:58.370Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-c2e81bcdca0d44cf","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:58.376Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e174cd320bb9461717a56b51: start","job_status":"start","queue_duration_s":0.238063,"scheduling_latency_s":0.238097}2155vm-test-run-nixbot-gitlab> gitlab # [ 330.807451] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.648Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e174cd320bb9461717a56b51","created_at":"2026-06-29T21:59:58.370Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-c2e81bcdca0d44cf","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:59:58.376Z","job_size_bytes":3,"pid":2518,"sidekiq_tid":"ma5e","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e174cd320bb9461717a56b51: done: 0.034058 sec","job_status":"done","queue_duration_s":0.238063,"scheduling_latency_s":0.238097,"redis_calls":7,"redis_duration_s":0.002247,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034058,"completed_at":"2026-06-29T21:59:58.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156vm-test-run-nixbot-gitlab> test script finished in 397.03s2157vm-test-run-nixbot-gitlab> cleanup2158vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2159vm-test-run-nixbot-gitlab> gitlab # [ 330.855679] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-06-29T21:59:58.650Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"aa3481a8a224eca0becf78c1","created_at":"2026-06-29T21:59:58.389Z","trace_propagation_headers":{"sentry-trace":"1d77a5145a9e40ecb32bd52bc2f0c97c-c2e81bcdca0d44cf","baggage":"sentry-trace_id=1d77a5145a9e40ecb32bd52bc2f0c97c,sentry-release=Unknown"},"correlation_id":"01KWAPAVKK6ENW6Q8TK9K5N1E7","mqemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13)2160vm-test-run-nixbot-gitlab> gitlab # e2161vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.40 seconds)21622163post-build step Upload coverage to codecov: ok2164Skipping codecov: project=Mic92/nixbot attr=aarch64-linux.nixbot-gitlab