nixbot

1these 14 derivations will be built:2  /nix/store/bzr2kq71qsg0rcfydwwvjcl6ynygdd2x-python3.13-nixbot.drv3  /nix/store/sa300sxfm2rgr7qkrbq34qzr0ganaaqb-nixbot-effects.drv4  /nix/store/aapig7pnl24crzymcbhly1k9cvnj9426-unit-nixbot.service.drv5  /nix/store/1rmsir9fvagvv7p36bz73nd10n0bcgll-system-units.drv6  /nix/store/ifa5mhi2rggyvw73n55l91f6n5qkbvcj-etc.drv7  /nix/store/92bzk2r3ky2n292qpdxs7hwyjqapcdjq-activate.drv8  /nix/store/a3sm20kfizhfma4gh8ng70hsd6d8z9z3-nixos-system-gitlab-test.drv9  /nix/store/yz5mfr6lv17v92hsc14r6s4zjjdyvrvw-closure-info.drv10  /nix/store/f7g05h9dv6icb9ximx34m3fxmlsdw1c3-closure-info.drv11  /nix/store/zfywpa5g83j72dlwqbsd7kg85ngi187z-run-nixos-vm.drv12  /nix/store/rkk7l3fyl9zwlwnyk4zg67wm7cb3bpjp-nixos-vm.drv13  /nix/store/3s2hsvi7n2385y6wvj7n5dfpnxnn5vxx-driverConfiguration.json.drv14  /nix/store/irnvm68alkdaf5nhb7js5gz51y2bagrv-nixos-test-driver-nixbot-gitlab.drv15  /nix/store/hfffrlfd5k69qx5bb5d4is908pdfq6zn-vm-test-run-nixbot-gitlab.drv16building '/nix/store/sa300sxfm2rgr7qkrbq34qzr0ganaaqb-nixbot-effects.drv' on 'ssh-ng://nix@eliza'17building '/nix/store/sa300sxfm2rgr7qkrbq34qzr0ganaaqb-nixbot-effects.drv'18nixbot-effects> Sourcing python-remove-tests-dir-hook19nixbot-effects> Sourcing python-catch-conflicts-hook.sh20nixbot-effects> Sourcing python-remove-bin-bytecode-hook.sh21nixbot-effects> Sourcing pypa-build-hook22nixbot-effects> Using pypaBuildPhase23nixbot-effects> Sourcing python-runtime-deps-check-hook24nixbot-effects> Using pythonRuntimeDepsCheckHook25nixbot-effects> Sourcing pypa-install-hook26nixbot-effects> Using pypaInstallPhase27nixbot-effects> Sourcing python-imports-check-hook.sh28nixbot-effects> Using pythonImportsCheckPhase29nixbot-effects> Sourcing python-namespaces-hook30nixbot-effects> Sourcing python-catch-conflicts-hook.sh31nixbot-effects> Running phase: unpackPhase32nixbot-effects> unpacking source archive /nix/store/i46gmc999gq1ln6iimynl8jg1b4crmb8-nixbot_effects33nixbot-effects> source root is nixbot_effects34nixbot-effects> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot_effects/pyproject.toml"35nixbot-effects> Running phase: patchPhase36nixbot-effects> Running phase: updateAutotoolsGnuConfigScriptsPhase37nixbot-effects> Running phase: configurePhase38nixbot-effects> no configure script, doing nothing39nixbot-effects> Running phase: buildPhase40nixbot-effects> Executing pypaBuildPhase41nixbot-effects> Creating a wheel...42nixbot-effects> pypa build flags: --no-isolation --outdir dist/ --wheel43nixbot-effects> * Getting build dependencies for wheel...44nixbot-effects> * Building wheel...45nixbot-effects> Successfully built nixbot_effects-0.0.1-py3-none-any.whl46nixbot-effects> Finished creating a wheel...47nixbot-effects> Finished executing pypaBuildPhase48nixbot-effects> Running phase: pythonRuntimeDepsCheckHook49nixbot-effects> Executing pythonRuntimeDepsCheck50nixbot-effects> Checking runtime dependencies for nixbot_effects-0.0.1-py3-none-any.whl51nixbot-effects> Finished executing pythonRuntimeDepsCheck52nixbot-effects> Running phase: installPhase53nixbot-effects> Executing pypaInstallPhase54nixbot-effects> Successfully installed nixbot_effects-0.0.1-py3-none-any.whl55nixbot-effects> Finished executing pypaInstallPhase56nixbot-effects> Running phase: pythonOutputDistPhase57nixbot-effects> Executing pythonOutputDistPhase58nixbot-effects> Finished executing pythonOutputDistPhase59nixbot-effects> Running phase: fixupPhase60nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects61nixbot-effects> checking for references to /build/ in /nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects...62nixbot-effects> patching script interpreter paths in /nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects63nixbot-effects> stripping (with command strip and flags -S -p) in  /nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects/lib /nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects/bin64nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/pfa5fz01dafhd5pl1i380hrr12axdxm3-nixbot-effects-dist65nixbot-effects> checking for references to /build/ in /nix/store/pfa5fz01dafhd5pl1i380hrr12axdxm3-nixbot-effects-dist...66nixbot-effects> patching script interpreter paths in /nix/store/pfa5fz01dafhd5pl1i380hrr12axdxm3-nixbot-effects-dist67nixbot-effects> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.1368nixbot-effects> wrapping `/nix/store/5bx9jbpzi6z82vjsvcsvxbn4wzaj33xy-nixbot-effects/bin/nixbot-effects'...69nixbot-effects> Executing pythonRemoveTestsDir70nixbot-effects> Finished executing pythonRemoveTestsDir71nixbot-effects> Running phase: installCheckPhase72nixbot-effects> no Makefile or custom installCheckPhase, doing nothing73nixbot-effects> Running phase: pythonCatchConflictsPhase74nixbot-effects> Running phase: pythonRemoveBinBytecodePhase75nixbot-effects> Running phase: pythonImportsCheckPhase76nixbot-effects> Executing pythonImportsCheckPhase77building '/nix/store/aapig7pnl24crzymcbhly1k9cvnj9426-unit-nixbot.service.drv' on 'ssh-ng://nix@eliza'78building '/nix/store/aapig7pnl24crzymcbhly1k9cvnj9426-unit-nixbot.service.drv'79building '/nix/store/1rmsir9fvagvv7p36bz73nd10n0bcgll-system-units.drv' on 'ssh-ng://nix@eliza'80building '/nix/store/1rmsir9fvagvv7p36bz73nd10n0bcgll-system-units.drv'81building '/nix/store/ifa5mhi2rggyvw73n55l91f6n5qkbvcj-etc.drv' on 'ssh-ng://nix@eliza'82building '/nix/store/ifa5mhi2rggyvw73n55l91f6n5qkbvcj-etc.drv'83building '/nix/store/92bzk2r3ky2n292qpdxs7hwyjqapcdjq-activate.drv' on 'ssh-ng://nix@eliza'84building '/nix/store/92bzk2r3ky2n292qpdxs7hwyjqapcdjq-activate.drv'85building '/nix/store/a3sm20kfizhfma4gh8ng70hsd6d8z9z3-nixos-system-gitlab-test.drv' on 'ssh-ng://nix@eliza'86building '/nix/store/a3sm20kfizhfma4gh8ng70hsd6d8z9z3-nixos-system-gitlab-test.drv'87nixos-system-gitlab-test> structuredAttrs is enabled88building '/nix/store/yz5mfr6lv17v92hsc14r6s4zjjdyvrvw-closure-info.drv' on 'ssh-ng://nix@eliza'89building '/nix/store/yz5mfr6lv17v92hsc14r6s4zjjdyvrvw-closure-info.drv'90closure-info> structuredAttrs is enabled91building '/nix/store/f7g05h9dv6icb9ximx34m3fxmlsdw1c3-closure-info.drv' on 'ssh-ng://nix@eliza'92building '/nix/store/f7g05h9dv6icb9ximx34m3fxmlsdw1c3-closure-info.drv'93closure-info> structuredAttrs is enabled94building '/nix/store/zfywpa5g83j72dlwqbsd7kg85ngi187z-run-nixos-vm.drv' on 'ssh-ng://nix@eliza'95building '/nix/store/zfywpa5g83j72dlwqbsd7kg85ngi187z-run-nixos-vm.drv'96building '/nix/store/rkk7l3fyl9zwlwnyk4zg67wm7cb3bpjp-nixos-vm.drv' on 'ssh-ng://nix@eliza'97building '/nix/store/rkk7l3fyl9zwlwnyk4zg67wm7cb3bpjp-nixos-vm.drv'98building '/nix/store/3s2hsvi7n2385y6wvj7n5dfpnxnn5vxx-driverConfiguration.json.drv' on 'ssh-ng://nix@eliza'99building '/nix/store/3s2hsvi7n2385y6wvj7n5dfpnxnn5vxx-driverConfiguration.json.drv'100driverConfiguration.json> structuredAttrs is enabled101building '/nix/store/irnvm68alkdaf5nhb7js5gz51y2bagrv-nixos-test-driver-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'102building '/nix/store/irnvm68alkdaf5nhb7js5gz51y2bagrv-nixos-test-driver-nixbot-gitlab.drv'103nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck105nixos-test-driver-nixbot-gitlab> All checks passed!106nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)107nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint108nixos-test-driver-nixbot-gitlab> All checks passed!109building '/nix/store/hfffrlfd5k69qx5bb5d4is908pdfq6zn-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'110building '/nix/store/hfffrlfd5k69qx5bb5d4is908pdfq6zn-vm-test-run-nixbot-gitlab.drv'111vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state112vm-test-run-nixbot-gitlab> start all VLans113vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)114vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds115vm-test-run-nixbot-gitlab> run the VM test script116vm-test-run-nixbot-gitlab> additionally exposed symbols:117vm-test-run-nixbot-gitlab>     gitlab,118vm-test-run-nixbot-gitlab>     vlan1,119vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh120vm-test-run-nixbot-gitlab> start all VMs121vm-test-run-nixbot-gitlab> gitlab: starting vm122vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)123vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...124vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.yuQfHtWzhs', fmt=raw size=1073741824125vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)126vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done127vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes128vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 96be7e04-e448-4d1a-9384-e709bbf319b3129vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:130vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376131vm-test-run-nixbot-gitlab> gitlab # 132vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done133vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done134vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done135vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.138vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...139vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.140vm-test-run-nixbot-gitlab> (finished: start all VMs, in 62.55 seconds)141vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service142vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting143vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]144vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun  1 15:51:08 UTC 2026145vm-test-run-nixbot-gitlab> gitlab # [    0.000000] KASLR enabled146vm-test-run-nixbot-gitlab> gitlab # [    0.000000] random: crng init done147vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Machine model: linux,dummy-virt148vm-test-run-nixbot-gitlab> gitlab # [    0.000000] efi: UEFI not found.149vm-test-run-nixbot-gitlab> gitlab # [    0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT150vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]151vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]152vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Zone ranges:153vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA      [mem 0x0000000040000000-0x00000000ffffffff]154vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA32    empty155vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]156vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Device   empty157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Movable zone start for each node158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Early memory node ranges159vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   node   0: [mem 0x0000000040000000-0x00000001bfffffff]160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] cma: Reserved 32 MiB at 0x00000000fe000000162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: probing for conduit method from DT.163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: PSCIv1.3 detected in firmware.164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Using standard PSCI v0.2 function IDs165vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Trusted OS migration not required166vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: SMC Calling Convention v1.1167vm-test-run-nixbot-gitlab> gitlab # [    0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)168vm-test-run-nixbot-gitlab> gitlab # [    0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296169vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Detected PIPT I-cache on CPU0170vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)171vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: GICv3 CPU interface172vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-v4173vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-BHB174vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38175vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23176vm-test-run-nixbot-gitlab> gitlab # [    0.000000] alternatives: applying boot alternatives177vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/k0nh0swla3wsq1hxdmpkjj75ziggqhjn-nixos-system-gitlab-test/init regInfo=/nix/store/d9balld56azjr4asbk19wm2ldi0j3fxa-closure-info/registration console=ttyAMA0,115200n8 console=tty0178vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Unknown kernel command line parameters "regInfo=/nix/store/d9balld56azjr4asbk19wm2ldi0j3fxa-closure-info/registration", will be passed to user space.179vm-test-run-nixbot-gitlab> gitlab # [    0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes180vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)181vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)182vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: area num 4.183vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)184vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Fallback order for Node 0: 0185vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1572864186vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Policy zone: Normal187vm-test-run-nixbot-gitlab> gitlab # [    0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off188vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1189vm-test-run-nixbot-gitlab> gitlab # [    0.000000] allocated 12582912 bytes of page_ext190vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocating 74648 entries in 292 pages191vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocated 292 pages with 3 groups192vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Hierarchical RCU implementation.193vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU event tracing is enabled.194vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.195vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Trampoline variant of Tasks RCU enabled.196vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Rude variant of Tasks RCU enabled.197vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Tracing variant of Tasks RCU enabled.198vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.199vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4200vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.201vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.202vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.203vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0204vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 256 SPIs implemented205vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 0 Extended SPIs implemented206vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Root IRQ handler: gic_handle_irq207vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI208vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0209vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000210vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS [mem 0x08080000-0x0809ffff]211vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)212vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)213vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: using LPI property table @0x0000000100f30000214vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000215vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.216vm-test-run-nixbot-gitlab> gitlab # [    0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).217vm-test-run-nixbot-gitlab> gitlab # [    0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns218vm-test-run-nixbot-gitlab> gitlab # [    0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns219vm-test-run-nixbot-gitlab> gitlab # [    0.000027] arm-pv: using stolen time PV220vm-test-run-nixbot-gitlab> gitlab # [    0.000379] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)221vm-test-run-nixbot-gitlab> gitlab # [    0.000553] Console: colour dummy device 80x25222vm-test-run-nixbot-gitlab> gitlab # [    0.000559] printk: legacy console [tty0] enabled223vm-test-run-nixbot-gitlab> gitlab # [    0.000728] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)224vm-test-run-nixbot-gitlab> gitlab # [    0.000734] pid_max: default: 32768 minimum: 301225vm-test-run-nixbot-gitlab> gitlab # [    0.000812] LSM: initializing lsm=capability,landlock,yama,bpf,ima226vm-test-run-nixbot-gitlab> gitlab # [    0.000935] landlock: Up and running.227vm-test-run-nixbot-gitlab> gitlab # [    0.000938] Yama: becoming mindful.228vm-test-run-nixbot-gitlab> gitlab # [    0.001341] LSM support for eBPF active229vm-test-run-nixbot-gitlab> gitlab # [    0.001529] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)230vm-test-run-nixbot-gitlab> gitlab # [    0.001622] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)231vm-test-run-nixbot-gitlab> gitlab # [    0.002672] cacheinfo: Unable to detect cache hierarchy for CPU 0232vm-test-run-nixbot-gitlab> gitlab # [    0.003326] rcu: Hierarchical SRCU implementation.233vm-test-run-nixbot-gitlab> gitlab # [    0.003330] rcu: 	Max phase no-delay instances is 1000.234vm-test-run-nixbot-gitlab> gitlab # [    0.003508] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level235vm-test-run-nixbot-gitlab> gitlab # [    0.004502] fsl-mc MSI: its@8080000 domain created236vm-test-run-nixbot-gitlab> gitlab # [    0.004595] EFI services will not be available.237vm-test-run-nixbot-gitlab> gitlab # [    0.004749] smp: Bringing up secondary CPUs ...238vm-test-run-nixbot-gitlab> gitlab # [    0.005370] Detected PIPT I-cache on CPU1239vm-test-run-nixbot-gitlab> gitlab # [    0.005467] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000240vm-test-run-nixbot-gitlab> gitlab # [    0.005585] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000241vm-test-run-nixbot-gitlab> gitlab # [    0.005706] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]242vm-test-run-nixbot-gitlab> gitlab # [    0.006719] Detected PIPT I-cache on CPU2243vm-test-run-nixbot-gitlab> gitlab # [    0.006888] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000244vm-test-run-nixbot-gitlab> gitlab # [    0.007007] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000245vm-test-run-nixbot-gitlab> gitlab # [    0.007129] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]246vm-test-run-nixbot-gitlab> gitlab # [    0.008063] Detected PIPT I-cache on CPU3247vm-test-run-nixbot-gitlab> gitlab # [    0.008176] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000248vm-test-run-nixbot-gitlab> gitlab # [    0.008296] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000249vm-test-run-nixbot-gitlab> gitlab # [    0.008418] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]250vm-test-run-nixbot-gitlab> gitlab # [    0.009218] smp: Brought up 1 node, 4 CPUs251vm-test-run-nixbot-gitlab> gitlab # [    0.009244] SMP: Total of 4 processors activated.252vm-test-run-nixbot-gitlab> gitlab # [    0.009247] CPU: All CPU(s) started at EL1253vm-test-run-nixbot-gitlab> gitlab # [    0.009256] CPU features: detected: Branch Target Identification254vm-test-run-nixbot-gitlab> gitlab # [    0.009260] CPU features: detected: ARMv8.4 Translation Table Level255vm-test-run-nixbot-gitlab> gitlab # [    0.009263] CPU features: detected: Instruction cache invalidation not required for I/D coherence256vm-test-run-nixbot-gitlab> gitlab # [    0.009266] CPU features: detected: Data cache clean to the PoU not required for I/D coherence257vm-test-run-nixbot-gitlab> gitlab # [    0.009270] CPU features: detected: Common not Private translations258vm-test-run-nixbot-gitlab> gitlab # [    0.009273] CPU features: detected: CRC32 instructions259vm-test-run-nixbot-gitlab> gitlab # [    0.009276] CPU features: detected: Data cache clean to Point of Deep Persistence260vm-test-run-nixbot-gitlab> gitlab # [    0.009279] CPU features: detected: Data cache clean to Point of Persistence261vm-test-run-nixbot-gitlab> gitlab # [    0.009282] CPU features: detected: Data independent timing control (DIT)262vm-test-run-nixbot-gitlab> gitlab # [    0.009285] CPU features: detected: E0PD263vm-test-run-nixbot-gitlab> gitlab # [    0.009287] CPU features: detected: Enhanced Counter Virtualization264vm-test-run-nixbot-gitlab> gitlab # [    0.009290] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)265vm-test-run-nixbot-gitlab> gitlab # [    0.009293] CPU features: detected: Enhanced Virtualization Traps266vm-test-run-nixbot-gitlab> gitlab # [    0.009296] CPU features: detected: Fine Grained Traps267vm-test-run-nixbot-gitlab> gitlab # [    0.009300] CPU features: detected: Generic authentication (architected QARMA5 algorithm)268vm-test-run-nixbot-gitlab> gitlab # [    0.009304] CPU features: detected: RCpc load-acquire (LDAPR)269vm-test-run-nixbot-gitlab> gitlab # [    0.009306] CPU features: detected: LSE atomic instructions270vm-test-run-nixbot-gitlab> gitlab # [    0.009309] CPU features: detected: Privileged Access Never271vm-test-run-nixbot-gitlab> gitlab # [    0.009312] CPU features: detected: PMUv3272vm-test-run-nixbot-gitlab> gitlab # [    0.009314] CPU features: detected: RAS Extension Support273vm-test-run-nixbot-gitlab> gitlab # [    0.009317] CPU features: detected: RASv1p1 Extension Support274vm-test-run-nixbot-gitlab> gitlab # [    0.009320] CPU features: detected: Random Number Generator275vm-test-run-nixbot-gitlab> gitlab # [    0.009322] CPU features: detected: Speculation barrier (SB)276vm-test-run-nixbot-gitlab> gitlab # [    0.009325] CPU features: detected: Stage-2 Force Write-Back277vm-test-run-nixbot-gitlab> gitlab # [    0.009328] CPU features: detected: TLB range maintenance instructions278vm-test-run-nixbot-gitlab> gitlab # [    0.009332] CPU features: detected: Speculative Store Bypassing Safe (SSBS)279vm-test-run-nixbot-gitlab> gitlab # [    0.009459] alternatives: applying system-wide alternatives280vm-test-run-nixbot-gitlab> gitlab # [    0.012250] CPU features: detected: BBM Level 2 without TLB conflict abort281vm-test-run-nixbot-gitlab> gitlab # [    0.012481] Memory: 5965820K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved)282vm-test-run-nixbot-gitlab> gitlab # [    0.013833] devtmpfs: initialized283vm-test-run-nixbot-gitlab> gitlab # [    0.017617] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns284vm-test-run-nixbot-gitlab> gitlab # [    0.017656] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)285vm-test-run-nixbot-gitlab> gitlab # [    0.017715] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).286vm-test-run-nixbot-gitlab> gitlab # [    0.017880] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL287vm-test-run-nixbot-gitlab> gitlab # [    0.017884] 0 pages in range for non-PLT usage288vm-test-run-nixbot-gitlab> gitlab # [    0.017886] 508336 pages in range for PLT usage289vm-test-run-nixbot-gitlab> gitlab # [    0.018014] pinctrl core: initialized pinctrl subsystem290vm-test-run-nixbot-gitlab> gitlab # [    0.018806] DMI not present or invalid.291vm-test-run-nixbot-gitlab> gitlab # [    0.021894] NET: Registered PF_NETLINK/PF_ROUTE protocol family292vm-test-run-nixbot-gitlab> gitlab # [    0.024223] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations293vm-test-run-nixbot-gitlab> gitlab # [    0.024492] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations294vm-test-run-nixbot-gitlab> gitlab # [    0.026103] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations295vm-test-run-nixbot-gitlab> gitlab # [    0.026123] audit: initializing netlink subsys (disabled)296vm-test-run-nixbot-gitlab> gitlab # [    0.026412] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1297vm-test-run-nixbot-gitlab> gitlab # [    0.026834] thermal_sys: Registered thermal governor 'fair_share'298vm-test-run-nixbot-gitlab> gitlab # [    0.026836] thermal_sys: Registered thermal governor 'bang_bang'299vm-test-run-nixbot-gitlab> gitlab # [    0.026840] thermal_sys: Registered thermal governor 'step_wise'300vm-test-run-nixbot-gitlab> gitlab # [    0.026843] thermal_sys: Registered thermal governor 'user_space'301vm-test-run-nixbot-gitlab> gitlab # [    0.026846] thermal_sys: Registered thermal governor 'power_allocator'302vm-test-run-nixbot-gitlab> gitlab # [    0.026924] cpuidle: using governor ladder303vm-test-run-nixbot-gitlab> gitlab # [    0.026947] cpuidle: using governor menu304vm-test-run-nixbot-gitlab> gitlab # [    0.027142] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.305vm-test-run-nixbot-gitlab> gitlab # [    0.027350] ASID allocator initialised with 65536 entries306vm-test-run-nixbot-gitlab> gitlab # [    0.028458] Serial: AMBA PL011 UART driver307vm-test-run-nixbot-gitlab> gitlab # [    0.033458] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1308vm-test-run-nixbot-gitlab> gitlab # [    0.033745] printk: console [ttyAMA0] enabled309vm-test-run-nixbot-gitlab> gitlab # [    0.038503] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages310vm-test-run-nixbot-gitlab> gitlab # [    0.038525] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page311vm-test-run-nixbot-gitlab> gitlab # [    0.038534] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages312vm-test-run-nixbot-gitlab> gitlab # [    0.038542] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page313vm-test-run-nixbot-gitlab> gitlab # [    0.038551] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages314vm-test-run-nixbot-gitlab> gitlab # [    0.038559] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page315vm-test-run-nixbot-gitlab> gitlab # [    0.038567] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages316vm-test-run-nixbot-gitlab> gitlab # [    0.038575] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page317vm-test-run-nixbot-gitlab> gitlab # [    0.048512] fbcon: Taking over console318vm-test-run-nixbot-gitlab> gitlab # [    0.048522] ACPI: Interpreter disabled.319vm-test-run-nixbot-gitlab> gitlab # [    0.049520] iommu: Default domain type: Translated320vm-test-run-nixbot-gitlab> gitlab # [    0.049535] iommu: DMA domain TLB invalidation policy: strict mode321vm-test-run-nixbot-gitlab> gitlab # [    0.065887] SCSI subsystem initialized322vm-test-run-nixbot-gitlab> gitlab # [    0.066753] usbcore: registered new interface driver usbfs323vm-test-run-nixbot-gitlab> gitlab # [    0.066839] usbcore: registered new interface driver hub324vm-test-run-nixbot-gitlab> gitlab # [    0.066884] usbcore: registered new device driver usb325vm-test-run-nixbot-gitlab> gitlab # [    0.067855] pps_core: LinuxPPS API ver. 1 registered326vm-test-run-nixbot-gitlab> gitlab # [    0.067867] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>327vm-test-run-nixbot-gitlab> gitlab # [    0.067915] PTP clock support registered328vm-test-run-nixbot-gitlab> gitlab # [    0.068049] EDAC MC: Ver: 3.0.0329vm-test-run-nixbot-gitlab> gitlab # [    0.068769] scmi_core: SCMI protocol bus registered330vm-test-run-nixbot-gitlab> gitlab # [    0.069985] FPGA manager framework331vm-test-run-nixbot-gitlab> gitlab # [    0.072528] vgaarb: loaded332vm-test-run-nixbot-gitlab> gitlab # [    0.073747] clocksource: Switched to clocksource arch_sys_counter333vm-test-run-nixbot-gitlab> gitlab # [    0.074888] VFS: Disk quotas dquot_6.6.0334vm-test-run-nixbot-gitlab> gitlab # [    0.074948] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)335vm-test-run-nixbot-gitlab> gitlab # [    0.075758] netfs: FS-Cache loaded336vm-test-run-nixbot-gitlab> gitlab # [    0.075989] pnp: PnP ACPI: disabled337vm-test-run-nixbot-gitlab> gitlab # [    0.090799] NET: Registered PF_INET protocol family338vm-test-run-nixbot-gitlab> gitlab # [    0.092956] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)339vm-test-run-nixbot-gitlab> gitlab # [    0.149590] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)340vm-test-run-nixbot-gitlab> gitlab # [    0.149661] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)341vm-test-run-nixbot-gitlab> gitlab # [    0.149721] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [    0.150185] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)343vm-test-run-nixbot-gitlab> gitlab # [    0.151106] TCP: Hash tables configured (established 65536 bind 65536)344vm-test-run-nixbot-gitlab> gitlab # [    0.151253] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)345vm-test-run-nixbot-gitlab> gitlab # [    0.151356] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)346vm-test-run-nixbot-gitlab> gitlab # [    0.151532] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)347vm-test-run-nixbot-gitlab> gitlab # [    0.151759] NET: Registered PF_UNIX/PF_LOCAL protocol family348vm-test-run-nixbot-gitlab> gitlab # [    0.151800] NET: Registered PF_XDP protocol family349vm-test-run-nixbot-gitlab> gitlab # [    0.151820] PCI: CLS 0 bytes, default 64350vm-test-run-nixbot-gitlab> gitlab # [    0.152165] Trying to unpack rootfs image as initramfs...351vm-test-run-nixbot-gitlab> gitlab # [    0.165876] kvm [1]: HYP mode not available352vm-test-run-nixbot-gitlab> gitlab # [    0.240002] Initialise system trusted keyrings353vm-test-run-nixbot-gitlab> gitlab # [    0.240279] workingset: timestamp_bits=42 max_order=21 bucket_order=0354vm-test-run-nixbot-gitlab> gitlab # [    0.242555] squashfs: version 4.0 (2009/01/31) Phillip Lougher355vm-test-run-nixbot-gitlab> gitlab # [    0.242755] 9p: Installing v9fs 9p2000 file system support356vm-test-run-nixbot-gitlab> gitlab # [    0.279593] Key type asymmetric registered357vm-test-run-nixbot-gitlab> gitlab # [    0.279623] Asymmetric key parser 'x509' registered358vm-test-run-nixbot-gitlab> gitlab # [    0.279913] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)359vm-test-run-nixbot-gitlab> gitlab # [    0.280350] io scheduler mq-deadline registered360vm-test-run-nixbot-gitlab> gitlab # [    0.280365] io scheduler kyber registered361vm-test-run-nixbot-gitlab> gitlab # [    0.297490] pl061_gpio 9030000.pl061: PL061 GPIO chip registered362vm-test-run-nixbot-gitlab> gitlab # [    0.301863] ledtrig-cpu: registered to indicate activity on CPUs363vm-test-run-nixbot-gitlab> gitlab # [    0.302920] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:364vm-test-run-nixbot-gitlab> gitlab # [    0.302974] pci-host-generic 4010000000.pcie:       IO 0x003eff0000..0x003effffff -> 0x0000000000365vm-test-run-nixbot-gitlab> gitlab # [    0.303021] pci-host-generic 4010000000.pcie:      MEM 0x0010000000..0x003efeffff -> 0x0010000000366vm-test-run-nixbot-gitlab> gitlab # [    0.303041] pci-host-generic 4010000000.pcie:      MEM 0x8000000000..0xffffffffff -> 0x8000000000367vm-test-run-nixbot-gitlab> gitlab # [    0.303084] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits368vm-test-run-nixbot-gitlab> gitlab # [    0.303134] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]369vm-test-run-nixbot-gitlab> gitlab # [    0.303341] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00370vm-test-run-nixbot-gitlab> gitlab # [    0.303361] pci_bus 0000:00: root bus resource [bus 00-ff]371vm-test-run-nixbot-gitlab> gitlab # [    0.303372] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]372vm-test-run-nixbot-gitlab> gitlab # [    0.303381] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]373vm-test-run-nixbot-gitlab> gitlab # [    0.303391] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]374vm-test-run-nixbot-gitlab> gitlab # [    0.303638] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint375vm-test-run-nixbot-gitlab> gitlab # [    0.304987] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint376vm-test-run-nixbot-gitlab> gitlab # [    0.305562] pci 0000:00:01.0: BAR 0 [io  0x0000-0x001f]377vm-test-run-nixbot-gitlab> gitlab # [    0.305597] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]378vm-test-run-nixbot-gitlab> gitlab # [    0.305686] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]379vm-test-run-nixbot-gitlab> gitlab # [    0.305721] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]380vm-test-run-nixbot-gitlab> gitlab # [    0.307286] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint381vm-test-run-nixbot-gitlab> gitlab # [    0.307878] pci 0000:00:02.0: BAR 0 [io  0x0000-0x001f]382vm-test-run-nixbot-gitlab> gitlab # [    0.307912] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]383vm-test-run-nixbot-gitlab> gitlab # [    0.308000] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]384vm-test-run-nixbot-gitlab> gitlab # [    0.309474] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint385vm-test-run-nixbot-gitlab> gitlab # [    0.310081] pci 0000:00:03.0: BAR 0 [io  0x0000-0x001f]386vm-test-run-nixbot-gitlab> gitlab # [    0.310116] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]387vm-test-run-nixbot-gitlab> gitlab # [    0.310203] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]388vm-test-run-nixbot-gitlab> gitlab # [    0.311683] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint389vm-test-run-nixbot-gitlab> gitlab # [    0.312261] pci 0000:00:04.0: BAR 0 [io  0x0000-0x001f]390vm-test-run-nixbot-gitlab> gitlab # [    0.312295] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]391vm-test-run-nixbot-gitlab> gitlab # [    0.312370] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]392vm-test-run-nixbot-gitlab> gitlab # [    0.313877] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint393vm-test-run-nixbot-gitlab> gitlab # [    0.314475] pci 0000:00:05.0: BAR 0 [io  0x0000-0x007f]394vm-test-run-nixbot-gitlab> gitlab # [    0.314508] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]395vm-test-run-nixbot-gitlab> gitlab # [    0.314583] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]396vm-test-run-nixbot-gitlab> gitlab # [    0.316072] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint397vm-test-run-nixbot-gitlab> gitlab # [    0.316631] pci 0000:00:06.0: BAR 0 [io  0x0000-0x007f]398vm-test-run-nixbot-gitlab> gitlab # [    0.316664] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]399vm-test-run-nixbot-gitlab> gitlab # [    0.316749] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]400vm-test-run-nixbot-gitlab> gitlab # [    0.318234] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint401vm-test-run-nixbot-gitlab> gitlab # [    0.318833] pci 0000:00:07.0: BAR 0 [io  0x0000-0x001f]402vm-test-run-nixbot-gitlab> gitlab # [    0.318867] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]403vm-test-run-nixbot-gitlab> gitlab # [    0.318941] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]404vm-test-run-nixbot-gitlab> gitlab # [    0.318975] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]405vm-test-run-nixbot-gitlab> gitlab # [    0.320538] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint406vm-test-run-nixbot-gitlab> gitlab # [    0.321115] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]407vm-test-run-nixbot-gitlab> gitlab # [    0.321190] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]408vm-test-run-nixbot-gitlab> gitlab # [    0.322724] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint409vm-test-run-nixbot-gitlab> gitlab # [    0.323320] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]410vm-test-run-nixbot-gitlab> gitlab # [    0.323396] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]411vm-test-run-nixbot-gitlab> gitlab # [    0.324689] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint412vm-test-run-nixbot-gitlab> gitlab # [    0.325256] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]413vm-test-run-nixbot-gitlab> gitlab # [    0.326039] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint414vm-test-run-nixbot-gitlab> gitlab # [    0.326711] pci 0000:00:0b.0: BAR 0 [io  0x0000-0x003f]415vm-test-run-nixbot-gitlab> gitlab # [    0.326745] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]416vm-test-run-nixbot-gitlab> gitlab # [    0.326832] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]417vm-test-run-nixbot-gitlab> gitlab # [    0.328239] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint418vm-test-run-nixbot-gitlab> gitlab # [    0.328761] pci 0000:00:0c.0: BAR 0 [io  0x0000-0x001f]419vm-test-run-nixbot-gitlab> gitlab # [    0.328795] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]420vm-test-run-nixbot-gitlab> gitlab # [    0.328869] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]421vm-test-run-nixbot-gitlab> gitlab # [    0.330577] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned422vm-test-run-nixbot-gitlab> gitlab # [    0.330595] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned423vm-test-run-nixbot-gitlab> gitlab # [    0.330606] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned424vm-test-run-nixbot-gitlab> gitlab # [    0.330725] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned425vm-test-run-nixbot-gitlab> gitlab # [    0.330865] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned426vm-test-run-nixbot-gitlab> gitlab # [    0.330986] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned427vm-test-run-nixbot-gitlab> gitlab # [    0.331120] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned428vm-test-run-nixbot-gitlab> gitlab # [    0.331249] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned429vm-test-run-nixbot-gitlab> gitlab # [    0.331393] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned430vm-test-run-nixbot-gitlab> gitlab # [    0.331516] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned431vm-test-run-nixbot-gitlab> gitlab # [    0.331651] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned432vm-test-run-nixbot-gitlab> gitlab # [    0.331774] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned433vm-test-run-nixbot-gitlab> gitlab # [    0.331933] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned434vm-test-run-nixbot-gitlab> gitlab # [    0.332055] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned435vm-test-run-nixbot-gitlab> gitlab # [    0.332120] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned436vm-test-run-nixbot-gitlab> gitlab # [    0.332170] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned437vm-test-run-nixbot-gitlab> gitlab # [    0.332220] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned438vm-test-run-nixbot-gitlab> gitlab # [    0.332283] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned439vm-test-run-nixbot-gitlab> gitlab # [    0.332334] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned440vm-test-run-nixbot-gitlab> gitlab # [    0.332384] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned441vm-test-run-nixbot-gitlab> gitlab # [    0.332446] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned442vm-test-run-nixbot-gitlab> gitlab # [    0.332499] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned443vm-test-run-nixbot-gitlab> gitlab # [    0.332551] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned444vm-test-run-nixbot-gitlab> gitlab # [    0.332603] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned445vm-test-run-nixbot-gitlab> gitlab # [    0.332656] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned446vm-test-run-nixbot-gitlab> gitlab # [    0.332707] pci 0000:00:05.0: BAR 0 [io  0x1000-0x107f]: assigned447vm-test-run-nixbot-gitlab> gitlab # [    0.332769] pci 0000:00:06.0: BAR 0 [io  0x1080-0x10ff]: assigned448vm-test-run-nixbot-gitlab> gitlab # [    0.332818] pci 0000:00:0b.0: BAR 0 [io  0x1100-0x113f]: assigned449vm-test-run-nixbot-gitlab> gitlab # [    0.332870] pci 0000:00:01.0: BAR 0 [io  0x1140-0x115f]: assigned450vm-test-run-nixbot-gitlab> gitlab # [    0.332932] pci 0000:00:02.0: BAR 0 [io  0x1160-0x117f]: assigned451vm-test-run-nixbot-gitlab> gitlab # [    0.332984] pci 0000:00:03.0: BAR 0 [io  0x1180-0x119f]: assigned452vm-test-run-nixbot-gitlab> gitlab # [    0.333034] pci 0000:00:04.0: BAR 0 [io  0x11a0-0x11bf]: assigned453vm-test-run-nixbot-gitlab> gitlab # [    0.333096] pci 0000:00:07.0: BAR 0 [io  0x11c0-0x11df]: assigned454vm-test-run-nixbot-gitlab> gitlab # [    0.333147] pci 0000:00:0c.0: BAR 0 [io  0x11e0-0x11ff]: assigned455vm-test-run-nixbot-gitlab> gitlab # [    0.333210] pci_bus 0000:00: resource 4 [io  0x0000-0xffff]456vm-test-run-nixbot-gitlab> gitlab # [    0.333224] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]457vm-test-run-nixbot-gitlab> gitlab # [    0.333233] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]458vm-test-run-nixbot-gitlab> gitlab # [    0.336822] pci 0000:00:0a.0: enabling device (0000 -> 0002)459vm-test-run-nixbot-gitlab> gitlab # [    0.370702] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)460vm-test-run-nixbot-gitlab> gitlab # [    0.375465] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)461vm-test-run-nixbot-gitlab> gitlab # [    0.379977] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)462vm-test-run-nixbot-gitlab> gitlab # [    0.385625] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)463vm-test-run-nixbot-gitlab> gitlab # [    0.390231] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)464vm-test-run-nixbot-gitlab> gitlab # [    0.395513] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)465vm-test-run-nixbot-gitlab> gitlab # [    0.400683] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)466vm-test-run-nixbot-gitlab> gitlab # [    0.405300] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)467vm-test-run-nixbot-gitlab> gitlab # [    0.410673] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)468vm-test-run-nixbot-gitlab> gitlab # [    0.415221] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)469vm-test-run-nixbot-gitlab> gitlab # [    0.421004] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)470vm-test-run-nixbot-gitlab> gitlab # [    0.433842] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled471vm-test-run-nixbot-gitlab> gitlab # [    0.437810] msm_serial: driver initialized472vm-test-run-nixbot-gitlab> gitlab # [    0.438234] SuperH (H)SCI(F) driver initialized473vm-test-run-nixbot-gitlab> gitlab # [    0.438344] STM32 USART driver initialized474vm-test-run-nixbot-gitlab> gitlab # [    0.476420] loop: module loaded475vm-test-run-nixbot-gitlab> gitlab # [    0.476717] virtio_blk virtio4: 4/0/0 default/read/poll queues476vm-test-run-nixbot-gitlab> gitlab # [    0.480029] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)477vm-test-run-nixbot-gitlab> gitlab # [    0.482826] virtio_blk virtio5: 4/0/0 default/read/poll queues478vm-test-run-nixbot-gitlab> gitlab # [    0.486050] virtio_blk virtio5: [vdb] 11826624 512-byte logical blocks (6.06 GB/5.64 GiB)479vm-test-run-nixbot-gitlab> gitlab # [    0.489330] megasas: 07.734.00.00-rc1480vm-test-run-nixbot-gitlab> gitlab # [    0.490711] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]481vm-test-run-nixbot-gitlab> gitlab # [    0.493805] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000482vm-test-run-nixbot-gitlab> gitlab # [    0.493832] Intel/Sharp Extended Query Table at 0x0031483vm-test-run-nixbot-gitlab> gitlab # [    0.496562] Using buffer write method484vm-test-run-nixbot-gitlab> gitlab # [    0.496645] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]485vm-test-run-nixbot-gitlab> gitlab # [    0.500106] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000486vm-test-run-nixbot-gitlab> gitlab # [    0.500130] Intel/Sharp Extended Query Table at 0x0031487vm-test-run-nixbot-gitlab> gitlab # [    0.503998] Using buffer write method488vm-test-run-nixbot-gitlab> gitlab # [    0.504019] Concatenating MTD devices:489vm-test-run-nixbot-gitlab> gitlab # [    0.504024] (0): "0.flash"490vm-test-run-nixbot-gitlab> gitlab # [    0.504027] (1): "0.flash"491vm-test-run-nixbot-gitlab> gitlab # [    0.504031] into device "0.flash"492vm-test-run-nixbot-gitlab> gitlab # [    0.578900] Freeing initrd memory: 25372K493vm-test-run-nixbot-gitlab> gitlab # [    0.592135] tun: Universal TUN/TAP device driver, 1.6494vm-test-run-nixbot-gitlab> gitlab # [    0.595727] thunder_xcv, ver 1.0495vm-test-run-nixbot-gitlab> gitlab # [    0.595774] thunder_bgx, ver 1.0496vm-test-run-nixbot-gitlab> gitlab # [    0.595803] nicpf, ver 1.0497vm-test-run-nixbot-gitlab> gitlab # [    0.596487] e1000: Intel(R) PRO/1000 Network Driver498vm-test-run-nixbot-gitlab> gitlab # [    0.596492] e1000: Copyright (c) 1999-2006 Intel Corporation.499vm-test-run-nixbot-gitlab> gitlab # [    0.596521] e1000e: Intel(R) PRO/1000 Network Driver500vm-test-run-nixbot-gitlab> gitlab # [    0.596527] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.501vm-test-run-nixbot-gitlab> gitlab # [    0.596555] igb: Intel(R) Gigabit Ethernet Network Driver502vm-test-run-nixbot-gitlab> gitlab # [    0.596559] igb: Copyright (c) 2007-2014 Intel Corporation.503vm-test-run-nixbot-gitlab> gitlab # [    0.596589] igbvf: Intel(R) Gigabit Virtual Function Network Driver504vm-test-run-nixbot-gitlab> gitlab # [    0.596594] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.505vm-test-run-nixbot-gitlab> gitlab # [    0.596746] sky2: driver version 1.30506vm-test-run-nixbot-gitlab> gitlab # [    0.598656] usbcore: registered new interface driver usb-storage507vm-test-run-nixbot-gitlab> gitlab # [    0.598777] usbcore: registered new interface driver usbserial_generic508vm-test-run-nixbot-gitlab> gitlab # [    0.598795] usbserial: USB Serial support registered for generic509vm-test-run-nixbot-gitlab> gitlab # [    0.599554] hv_vmbus: registering driver hyperv_keyboard510vm-test-run-nixbot-gitlab> gitlab # [    0.599635] ehci-pci 0000:00:0a.0: EHCI Host Controller511vm-test-run-nixbot-gitlab> gitlab # [    0.599678] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1512vm-test-run-nixbot-gitlab> gitlab # [    0.600225] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000513vm-test-run-nixbot-gitlab> gitlab # [    0.600769] rtc-pl031 9010000.pl031: registered as rtc0514vm-test-run-nixbot-gitlab> gitlab # [    0.600798] rtc-pl031 9010000.pl031: setting system clock to 2026-06-14T14:24:53 UTC (1781447093)515vm-test-run-nixbot-gitlab> gitlab # [    0.601161] i2c_dev: i2c /dev entries driver516vm-test-run-nixbot-gitlab> gitlab # [    0.604394] sdhci: Secure Digital Host Controller Interface driver517vm-test-run-nixbot-gitlab> gitlab # [    0.604401] sdhci: Copyright(c) Pierre Ossman518vm-test-run-nixbot-gitlab> gitlab # [    0.604697] Synopsys Designware Multimedia Card Interface Driver519vm-test-run-nixbot-gitlab> gitlab # [    0.605135] sdhci-pltfm: SDHCI platform and OF driver helper520vm-test-run-nixbot-gitlab> gitlab # [    0.607062] hid: raw HID events driver (C) Jiri Kosina521vm-test-run-nixbot-gitlab> gitlab # [    0.607478] usbcore: registered new interface driver usbhid522vm-test-run-nixbot-gitlab> gitlab # [    0.607484] usbhid: USB HID core driver523vm-test-run-nixbot-gitlab> gitlab # [    0.609907] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00524vm-test-run-nixbot-gitlab> gitlab # [    0.611259] hub 1-0:1.0: USB hub found525vm-test-run-nixbot-gitlab> gitlab # [    0.611364] hub 1-0:1.0: 6 ports detected526vm-test-run-nixbot-gitlab> gitlab # [    0.648339] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available527vm-test-run-nixbot-gitlab> gitlab # [    0.651034] drop_monitor: Initializing network drop monitor service528vm-test-run-nixbot-gitlab> gitlab # [    0.651220] NET: Registered PF_INET6 protocol family529vm-test-run-nixbot-gitlab> gitlab # [    0.653162] Segment Routing with IPv6530vm-test-run-nixbot-gitlab> gitlab # [    0.653197] In-situ OAM (IOAM) with IPv6531vm-test-run-nixbot-gitlab> gitlab # [    0.653338] NET: Registered PF_PACKET protocol family532vm-test-run-nixbot-gitlab> gitlab # [    0.653779] 9pnet: Installing 9P2000 support533vm-test-run-nixbot-gitlab> gitlab # [    0.657706] Key type dns_resolver registered534vm-test-run-nixbot-gitlab> gitlab # [    0.676788] registered taskstats version 1535vm-test-run-nixbot-gitlab> gitlab # [    0.677436] Loading compiled-in X.509 certificates536vm-test-run-nixbot-gitlab> gitlab # [    0.703879] Demotion targets for Node 0: null537vm-test-run-nixbot-gitlab> gitlab # [    0.704195] Key type .fscrypt registered538vm-test-run-nixbot-gitlab> gitlab # [    0.704205] Key type fscrypt-provisioning registered539vm-test-run-nixbot-gitlab> gitlab # [    0.704450] ima: No TPM chip found, activating TPM-bypass!540vm-test-run-nixbot-gitlab> gitlab # [    0.704480] ima: Allocated hash algorithm: sha1541vm-test-run-nixbot-gitlab> gitlab # [    0.704559] ima: No architecture policies found542vm-test-run-nixbot-gitlab> gitlab # [    0.706338] input: gpio-keys as /devices/platform/gpio-keys/input/input0543vm-test-run-nixbot-gitlab> gitlab # [    0.744894] clk: Disabling unused clocks544vm-test-run-nixbot-gitlab> gitlab # [    0.744940] PM: genpd: Disabling unused power domains545vm-test-run-nixbot-gitlab> gitlab # [    0.753528] Freeing unused kernel memory: 4736K546vm-test-run-nixbot-gitlab> gitlab # [    0.753884] Run /init as init process547vm-test-run-nixbot-gitlab> gitlab # [    0.796973] systemd[1]: Successfully made /usr/ read-only.548vm-test-run-nixbot-gitlab> gitlab # [    0.857862] usb 1-1: new high-speed USB device number 2 using ehci-pci549vm-test-run-nixbot-gitlab> gitlab # [    1.010341] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1550vm-test-run-nixbot-gitlab> gitlab # [    1.098275] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0551vm-test-run-nixbot-gitlab> gitlab # [    1.131883] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)552vm-test-run-nixbot-gitlab> gitlab # [    1.131950] systemd[1]: Detected virtualization qemu.553vm-test-run-nixbot-gitlab> gitlab # [    1.132011] systemd[1]: Detected architecture arm64.554vm-test-run-nixbot-gitlab> gitlab # [    1.132025] systemd[1]: Running in initrd.555vm-test-run-nixbot-gitlab> gitlab # [    1.133005] systemd[1]: Initializing machine ID from random generator.556vm-test-run-nixbot-gitlab> gitlab # [    1.133264] systemd[1]: Hostname set to <gitlab>.557vm-test-run-nixbot-gitlab> gitlab # [    1.217832] usb 1-2: new high-speed USB device number 3 using ehci-pci558vm-test-run-nixbot-gitlab> gitlab # [    1.285736] systemd[1]: Queued start job for default target Initrd Default Target.559vm-test-run-nixbot-gitlab> gitlab # [    1.372243] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2560vm-test-run-nixbot-gitlab> gitlab # [    1.373477] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0561vm-test-run-nixbot-gitlab> gitlab # [    1.532790] systemd[1]: Created slice Slice /system/modprobe.562vm-test-run-nixbot-gitlab> gitlab # [    1.533366] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.563vm-test-run-nixbot-gitlab> gitlab # [    1.533438] systemd[1]: Expecting device /dev/disk/by-label/nix-store...564vm-test-run-nixbot-gitlab> gitlab # [    1.533465] systemd[1]: Expecting device /dev/disk/by-label/nixos...565vm-test-run-nixbot-gitlab> gitlab # [    1.533527] systemd[1]: Reached target Path Units.566vm-test-run-nixbot-gitlab> gitlab # [    1.533564] systemd[1]: Reached target Slice Units.567vm-test-run-nixbot-gitlab> gitlab # [    1.533605] systemd[1]: Reached target Swaps.568vm-test-run-nixbot-gitlab> gitlab # [    1.533645] systemd[1]: Reached target Timer Units.569vm-test-run-nixbot-gitlab> gitlab # [    1.534132] systemd[1]: Listening on D-Bus System Message Bus Socket.570vm-test-run-nixbot-gitlab> gitlab # [    1.534484] systemd[1]: Listening on Journal Socket (/dev/log).571vm-test-run-nixbot-gitlab> gitlab # [    1.534802] systemd[1]: Listening on Journal Sockets.572vm-test-run-nixbot-gitlab> gitlab # [    1.535099] systemd[1]: Listening on udev Control Socket.573vm-test-run-nixbot-gitlab> gitlab # [    1.535276] systemd[1]: Listening on udev Kernel Socket.574vm-test-run-nixbot-gitlab> gitlab # [    1.535348] systemd[1]: Reached target Socket Units.575vm-test-run-nixbot-gitlab> gitlab # [    1.542726] systemd[1]: Starting Create List of Static Device Nodes...576vm-test-run-nixbot-gitlab> gitlab # [    1.550874] systemd[1]: Starting Load Kernel Module 9pnet_virtio...577vm-test-run-nixbot-gitlab> gitlab # [    1.551110] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs578vm-test-run-nixbot-gitlab> gitlab # [    1.559720] systemd[1]: Mounting Kernel Configuration File System...579vm-test-run-nixbot-gitlab> gitlab # [    1.582712] systemd[1]: Starting Journal Service...580vm-test-run-nixbot-gitlab> gitlab # [    1.592739] systemd[1]: Starting Load Kernel Modules...581vm-test-run-nixbot-gitlab> gitlab # [    1.593004] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki582vm-test-run-nixbot-gitlab> gitlab # [    1.601001] systemd[1]: Starting Coldplug All udev Devices...583vm-test-run-nixbot-gitlab> gitlab # [    1.622483] systemd[1]: Finished Create List of Static Device Nodes.584vm-test-run-nixbot-gitlab> gitlab # [    1.625832] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.585vm-test-run-nixbot-gitlab> gitlab # [    1.631988] systemd[1]: Finished Load Kernel Module 9pnet_virtio.586vm-test-run-nixbot-gitlab> gitlab # [    1.642785] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0587vm-test-run-nixbot-gitlab> gitlab # [    1.643265] [drm] features: -virgl +edid -resource_blob -host_visible588vm-test-run-nixbot-gitlab> gitlab # [    1.643272] [drm] features: -context_init589vm-test-run-nixbot-gitlab> gitlab # [    1.645379] systemd[1]: Mounted Kernel Configuration File System.590vm-test-run-nixbot-gitlab> gitlab # [    1.645554] systemd-journald[156]: Collecting audit messages is disabled.591vm-test-run-nixbot-gitlab> gitlab # [    1.647362] [drm] number of scanouts: 1592vm-test-run-nixbot-gitlab> gitlab # [    1.647388] [drm] number of cap sets: 0593vm-test-run-nixbot-gitlab> gitlab # [    1.658056] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.594vm-test-run-nixbot-gitlab> gitlab # [    1.661943] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...595vm-test-run-nixbot-gitlab> gitlab # [    1.662172] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev596vm-test-run-nixbot-gitlab> gitlab # [    1.669077] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic597vm-test-run-nixbot-gitlab> gitlab # [    1.669123] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0598vm-test-run-nixbot-gitlab> gitlab # [    1.684668] Console: switching to colour frame buffer device 160x50599vm-test-run-nixbot-gitlab> gitlab # [    1.691203] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device600vm-test-run-nixbot-gitlab> gitlab # [    1.715121] systemd[1]: Finished Load Kernel Modules.601vm-test-run-nixbot-gitlab> gitlab # [    1.717152] systemd-modules-load[158]: Using 4 probe threads602vm-test-run-nixbot-gitlab> gitlab # [    1.718312] systemd-modules-load[158]: Module 'virtio_balloon' is built in603vm-test-run-nixbot-gitlab> gitlab # [    1.719357] systemd-modules-load[158]: Module 'virtio_console' is built in604vm-test-run-nixbot-gitlab> gitlab # [    1.720502] systemd-modules-load[158]: Module 'virtio_rng' is built in605vm-test-run-nixbot-gitlab> gitlab # [    1.721518] systemd-modules-load[158]: Inserted module 'dm_mod'[    1.723442] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.606vm-test-run-nixbot-gitlab> gitlab # 607vm-test-run-nixbot-gitlab> gitlab # [    1.727111] systemd[1]: Started Journal Service.608vm-test-run-nixbot-gitlab> gitlab # [    1.727272] systemd-modules-load[158]: Inserted module 'virtio_gpu'609vm-test-run-nixbot-gitlab> gitlab # [    1.741608] systemd[1]: Starting Apply Kernel Variables...610vm-test-run-nixbot-gitlab> gitlab # [    1.752176] systemd[1]: Starting Create Static Device Nodes in /dev...611vm-test-run-nixbot-gitlab> gitlab # [    1.792800] systemd[1]: Finished Apply Kernel Variables.612vm-test-run-nixbot-gitlab> gitlab # [    1.798335] systemd[1]: Finished Create Static Device Nodes in /dev.613vm-test-run-nixbot-gitlab> gitlab # [    1.801010] systemd[1]: Reached target Preparation for Local File Systems.614vm-test-run-nixbot-gitlab> gitlab # [    1.803746] systemd[1]: Reached target Local File Systems.615vm-test-run-nixbot-gitlab> gitlab # [    1.807123] systemd[1]: Starting Create System Files and Directories...616vm-test-run-nixbot-gitlab> gitlab # [    1.816570] systemd[1]: Starting Rule-based Manager for Device Events and Files...617vm-test-run-nixbot-gitlab> gitlab # [    1.851289] systemd[1]: Finished Create System Files and Directories.618vm-test-run-nixbot-gitlab> gitlab # [    1.868418] systemd-udevd[193]: Using default interface naming scheme 'v260'.619vm-test-run-nixbot-gitlab> gitlab # [    1.891680] systemd[1]: Started Rule-based Manager for Device Events and Files.620vm-test-run-nixbot-gitlab> gitlab # [    1.923554] systemd[1]: Starting Virtual Console Setup...621vm-test-run-nixbot-gitlab> gitlab # [    1.980466] systemd-vconsole-setup[210]: Configuration of first virtual console was skipped, ignoring remaining ones.622vm-test-run-nixbot-gitlab> gitlab # [    1.985933] systemd[1]: Finished Virtual Console Setup.623vm-test-run-nixbot-gitlab> gitlab # [    2.345490] systemd[1]: Finished Coldplug All udev Devices.624vm-test-run-nixbot-gitlab> gitlab # [    2.348081] systemd[1]: Reached target System Initialization.625vm-test-run-nixbot-gitlab> gitlab # [    2.350457] systemd[1]: Reached target Basic System.626vm-test-run-nixbot-gitlab> gitlab # [    2.539273] (udev-worker)[205]: Network interface NamePolicy= disabled on kernel command line.627vm-test-run-nixbot-gitlab> gitlab # [    2.555175] (udev-worker)[206]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.628vm-test-run-nixbot-gitlab> gitlab # [    2.560662] (udev-worker)[206]: Network interface NamePolicy= disabled on kernel command line.629vm-test-run-nixbot-gitlab> gitlab # [    2.566796] systemd[1]: Found device /dev/disk/by-label/nix-store.630vm-test-run-nixbot-gitlab> gitlab # [    2.595905] systemd[1]: Found device /dev/disk/by-label/nixos.631vm-test-run-nixbot-gitlab> gitlab # [    2.596990] systemd[1]: Reached target Initrd Root Device.632vm-test-run-nixbot-gitlab> gitlab # [    2.603183] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...633vm-test-run-nixbot-gitlab> gitlab # [    2.636457] systemd-fsck[235]: nixos: clean, 12/65536 files, 13019/262144 blocks634vm-test-run-nixbot-gitlab> gitlab # [    2.642293] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.635vm-test-run-nixbot-gitlab> gitlab # [    2.647485] systemd[1]: Mounting /sysroot...636vm-test-run-nixbot-gitlab> gitlab # [    2.685491] EXT4-fs (vda): mounted filesystem 96be7e04-e448-4d1a-9384-e709bbf319b3 r/w with ordered data mode. Quota mode: none.637vm-test-run-nixbot-gitlab> gitlab # [    2.685744] systemd[1]: Mounted /sysroot.638vm-test-run-nixbot-gitlab> gitlab # [    2.689213] systemd[1]: Reached target Initrd Root File System.639vm-test-run-nixbot-gitlab> gitlab # [    2.693308] systemd[1]: Starting Mountpoints Configured in the Real Root...640vm-test-run-nixbot-gitlab> gitlab # [    2.722324] systemd-sysroot-fstab-check[246]: /sysroot should be mounted in the initrd, will request daemon-reload.641vm-test-run-nixbot-gitlab> gitlab # [    2.725076] systemd[1]: Reload requested from client PID 246 ('systemd-sysroot') (unit initrd-parse-etc.service)...642vm-test-run-nixbot-gitlab> gitlab # [    2.727830] systemd[1]: Reloading...643vm-test-run-nixbot-gitlab> gitlab # [    3.031824] systemd[1]: Reloading finished in 305 ms.644vm-test-run-nixbot-gitlab> gitlab # [    3.061709] systemd-sysroot-fstab-check[246]: Requesting initrd-fs.target/start/replace...645vm-test-run-nixbot-gitlab> gitlab # [    3.154970] systemd-sysroot-fstab-check[246]: Requesting swap.target/start/replace...646vm-test-run-nixbot-gitlab> gitlab # [    3.160877] systemd[1]: Starting Load Kernel Module 9pnet_virtio...647vm-test-run-nixbot-gitlab> gitlab # [    3.161787] systemd[1]: initrd-parse-etc.service: Deactivated successfully.648vm-test-run-nixbot-gitlab> gitlab # [    3.163811] systemd[1]: Finished Mountpoints Configured in the Real Root.649vm-test-run-nixbot-gitlab> gitlab # [    3.164933] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.650vm-test-run-nixbot-gitlab> gitlab # [    3.177171] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.651vm-test-run-nixbot-gitlab> gitlab # [    3.179162] systemd[1]: Finished Load Kernel Module 9pnet_virtio.652vm-test-run-nixbot-gitlab> gitlab # [    3.185926] (udev-worker)[226]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory653vm-test-run-nixbot-gitlab> gitlab # [    3.188901] (udev-worker)[226]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory654vm-test-run-nixbot-gitlab> gitlab # [    3.199506] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.655vm-test-run-nixbot-gitlab> gitlab # [    3.200622] systemd[1]: Stopped Virtual Console Setup.656vm-test-run-nixbot-gitlab> gitlab # [    3.202464] systemd[1]: Stopping Virtual Console Setup...657vm-test-run-nixbot-gitlab> gitlab # [    3.204926] systemd[1]: Starting Virtual Console Setup...658vm-test-run-nixbot-gitlab> gitlab # [    3.236326] systemd-vconsole-setup[455]: Configuration of first virtual console was skipped, ignoring remaining ones.659vm-test-run-nixbot-gitlab> gitlab # [    3.239519] systemd[1]: Finished Virtual Console Setup.660vm-test-run-nixbot-gitlab> gitlab # [    3.623468] systemd[1]: Mounting /sysroot/nix/.ro-store...661vm-test-run-nixbot-gitlab> gitlab # [    3.631148] systemd[1]: Mounting /sysroot/nix/.rw-store...662vm-test-run-nixbot-gitlab> gitlab # [    3.639513] systemd[1]: Mounting /sysroot/run...663vm-test-run-nixbot-gitlab> gitlab # [    3.656823] systemd[1]: Mounting /sysroot/tmp/shared...664vm-test-run-nixbot-gitlab> gitlab # [    3.672177] systemd[1]: Mounting /sysroot/tmp/xchg...665vm-test-run-nixbot-gitlab> gitlab # [    3.687855] erofs (device vdb): mounted with root inode @ nid 128.666vm-test-run-nixbot-gitlab> gitlab # [    3.700140] systemd[1]: Mounted /sysroot/nix/.ro-store.667vm-test-run-nixbot-gitlab> gitlab # [    3.702461] systemd[1]: Mounted /sysroot/nix/.rw-store.668vm-test-run-nixbot-gitlab> gitlab # [    3.704883] systemd[1]: Mounted /sysroot/run.669vm-test-run-nixbot-gitlab> gitlab # [    3.706774] systemd[1]: Mounted /sysroot/tmp/shared.670vm-test-run-nixbot-gitlab> gitlab # [    3.708894] systemd[1]: Mounted /sysroot/tmp/xchg.671vm-test-run-nixbot-gitlab> gitlab # [    3.715242] systemd[1]: Starting rw-sysroot-nix-store.service...672vm-test-run-nixbot-gitlab> gitlab # [    3.748291] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.673vm-test-run-nixbot-gitlab> gitlab # [    3.753238] systemd[1]: Finished rw-sysroot-nix-store.service.674vm-test-run-nixbot-gitlab> gitlab # [    3.770037] systemd[1]: Mounting /sysroot/nix/store...675vm-test-run-nixbot-gitlab> gitlab # [    3.836613] systemd[1]: Mounted /sysroot/nix/store.676vm-test-run-nixbot-gitlab> gitlab # [    3.838753] systemd[1]: Reached target Initrd File Systems.677vm-test-run-nixbot-gitlab> gitlab # [    3.844610] systemd[1]: Starting Find NixOS closure...678vm-test-run-nixbot-gitlab> gitlab # [    3.851952] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...679vm-test-run-nixbot-gitlab> gitlab # [    3.898809] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.680vm-test-run-nixbot-gitlab> gitlab # [    3.915284] systemd[1]: Finished Find NixOS closure.681vm-test-run-nixbot-gitlab> gitlab # [    3.917513] systemd[1]: Reached target Initrd Default Target.682vm-test-run-nixbot-gitlab> gitlab # [    3.923434] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...683vm-test-run-nixbot-gitlab> gitlab # [    3.968977] systemd[1]: Stopped target Initrd Default Target.684vm-test-run-nixbot-gitlab> gitlab # [    3.971388] systemd[1]: Stopped target Basic System.685vm-test-run-nixbot-gitlab> gitlab # [    3.973579] systemd[1]: Stopped target Initrd Root Device.686vm-test-run-nixbot-gitlab> gitlab # [    3.975813] systemd[1]: Stopped target Path Units.687vm-test-run-nixbot-gitlab> gitlab # [    3.977937] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.688vm-test-run-nixbot-gitlab> gitlab # [    3.980986] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.689vm-test-run-nixbot-gitlab> gitlab # [    3.984084] systemd[1]: Stopped target Slice Units.690vm-test-run-nixbot-gitlab> gitlab # [    3.986203] systemd[1]: Stopped target Socket Units.691vm-test-run-nixbot-gitlab> gitlab # [    3.988325] systemd[1]: Stopped target System Initialization.692vm-test-run-nixbot-gitlab> gitlab # [    3.990655] systemd[1]: Stopped target Swaps.693vm-test-run-nixbot-gitlab> gitlab # [    3.992561] systemd[1]: Stopped target Timer Units.694vm-test-run-nixbot-gitlab> gitlab # [    3.994636] systemd[1]: dbus.socket: Deactivated successfully.695vm-test-run-nixbot-gitlab> gitlab # [    3.997044] systemd[1]: Closed D-Bus System Message Bus Socket.696vm-test-run-nixbot-gitlab> gitlab # [    3.999391] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.697vm-test-run-nixbot-gitlab> gitlab # [    4.002519] systemd[1]: Stopped Find NixOS closure.698vm-test-run-nixbot-gitlab> gitlab # [    4.004832] systemd[1]: Starting Load Kernel Module 9pnet_virtio...699vm-test-run-nixbot-gitlab> gitlab # [    4.007378] systemd[1]: Starting rw-sysroot-nix-store.service...700vm-test-run-nixbot-gitlab> gitlab # [    4.009988] systemd[1]: systemd-sysctl.service: Deactivated successfully.701vm-test-run-nixbot-gitlab> gitlab # [    4.012712] systemd[1]: Stopped Apply Kernel Variables.702vm-test-run-nixbot-gitlab> gitlab # [    4.015816] systemd[1]: systemd-modules-load.service: Deactivated successfully.703vm-test-run-nixbot-gitlab> gitlab # [    4.018897] systemd[1]: Stopped Load Kernel Modules.704vm-test-run-nixbot-gitlab> gitlab # [    4.031032] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.705vm-test-run-nixbot-gitlab> gitlab # [    4.034509] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.706vm-test-run-nixbot-gitlab> gitlab # [    4.040509] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.707vm-test-run-nixbot-gitlab> gitlab # [    4.043427] systemd[1]: Stopped Create System Files and Directories.708vm-test-run-nixbot-gitlab> gitlab # [    4.049848] systemd[1]: Stopped target Local File Systems.709vm-test-run-nixbot-gitlab> gitlab # [    4.052144] systemd[1]: Stopped target Preparation for Local File Systems.710vm-test-run-nixbot-gitlab> gitlab # [    4.054851] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.711vm-test-run-nixbot-gitlab> gitlab # [    4.057826] systemd[1]: Stopped Coldplug All udev Devices.712vm-test-run-nixbot-gitlab> gitlab # [    4.060066] systemd[1]: Stopping Rule-based Manager for Device Events and Files...713vm-test-run-nixbot-gitlab> gitlab # [    4.063015] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [    4.066035] systemd[1]: Stopped Virtual Console Setup.715vm-test-run-nixbot-gitlab> gitlab # [    4.081237] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [    4.086255] systemd[1]: Finished Load Kernel Module 9pnet_virtio.717vm-test-run-nixbot-gitlab> gitlab # [    4.096657] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.718vm-test-run-nixbot-gitlab> gitlab # [    4.101599] systemd[1]: Finished rw-sysroot-nix-store.service.719vm-test-run-nixbot-gitlab> gitlab # [    4.111818] systemd[1]: systemd-udevd.service: Deactivated successfully.720vm-test-run-nixbot-gitlab> gitlab # [    4.117033] systemd[1]: Stopped Rule-based Manager for Device Events and Files.721vm-test-run-nixbot-gitlab> gitlab # [    4.119904] systemd[1]: systemd-udevd.service: Consumed 2.366s CPU time over 2.296s wall clock time, 32.9M memory peak.722vm-test-run-nixbot-gitlab> gitlab # [    4.136456] systemd[1]: initrd-cleanup.service: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [    4.141413] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.724vm-test-run-nixbot-gitlab> gitlab # [    4.155585] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.725vm-test-run-nixbot-gitlab> gitlab # [    4.158708] systemd[1]: Closed udev Control Socket.726vm-test-run-nixbot-gitlab> gitlab # [    4.171341] systemd[1]: Starting Cleanup udev Database...727vm-test-run-nixbot-gitlab> gitlab # [    4.173712] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.728vm-test-run-nixbot-gitlab> gitlab # [    4.176813] systemd[1]: Stopped Create Static Device Nodes in /dev.729vm-test-run-nixbot-gitlab> gitlab # [    4.179874] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [    4.183300] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.731vm-test-run-nixbot-gitlab> gitlab # [    4.188916] systemd[1]: kmod-static-nodes.service: Deactivated successfully.732vm-test-run-nixbot-gitlab> gitlab # [    4.191727] systemd[1]: Stopped Create List of Static Device Nodes.733vm-test-run-nixbot-gitlab> gitlab # [    4.268763] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.734vm-test-run-nixbot-gitlab> gitlab # [    4.271648] systemd[1]: Finished Cleanup udev Database.735vm-test-run-nixbot-gitlab> gitlab # [    4.278967] systemd[1]: Reached target Switch Root.736vm-test-run-nixbot-gitlab> gitlab # [    4.286729] systemd[1]: Starting NixOS Activation...737vm-test-run-nixbot-gitlab> gitlab # [    4.477258] initrd-nixos-activation-start[555]: booting system configuration /nix/store/k0nh0swla3wsq1hxdmpkjj75ziggqhjn-nixos-system-gitlab-test738vm-test-run-nixbot-gitlab> gitlab # [    4.538894] initrd-nixos-activation-start[555]: running activation script...739vm-test-run-nixbot-gitlab> gitlab # [    4.927979] initrd-nixos-activation-start[578]: setting up /etc...740vm-test-run-nixbot-gitlab> gitlab # [    5.102189] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.741vm-test-run-nixbot-gitlab> gitlab # [    5.106506] systemd[1]: Finished NixOS Activation.742vm-test-run-nixbot-gitlab> gitlab # [    5.122794] systemd[1]: Starting Switch Root...743vm-test-run-nixbot-gitlab> gitlab # [    5.179436] systemd[1]: Switching root.744vm-test-run-nixbot-gitlab> gitlab # [    5.750584] systemd-journald[156]: Received SIGTERM from PID 1 (systemd).745vm-test-run-nixbot-gitlab> gitlab # [    6.463210] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)746vm-test-run-nixbot-gitlab> gitlab # [    6.463363] systemd[1]: Detected virtualization qemu.747vm-test-run-nixbot-gitlab> gitlab # [    6.463450] systemd[1]: Detected architecture arm64.748vm-test-run-nixbot-gitlab> gitlab # [    6.463620] systemd[1]: Detected first boot.749vm-test-run-nixbot-gitlab> gitlab # [    6.465075] systemd[1]: Initializing machine ID from random generator.750vm-test-run-nixbot-gitlab> gitlab # [    6.692718] systemd[1]: bpf-restrict-fs: LSM BPF program attached751vm-test-run-nixbot-gitlab> gitlab # [    6.786690] systemd[1]: Applying preset policy.752vm-test-run-nixbot-gitlab> gitlab # [    7.005374] systemd[1]: Populated /etc with preset unit settings.753vm-test-run-nixbot-gitlab> gitlab # [    7.155691] systemd[1]: initrd-switch-root.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [    7.156094] systemd[1]: Stopped initrd-switch-root.service.755vm-test-run-nixbot-gitlab> gitlab # [    7.158251] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.756vm-test-run-nixbot-gitlab> gitlab # [    7.166424] systemd[1]: Created slice Slice /system/getty.757vm-test-run-nixbot-gitlab> gitlab # [    7.169176] systemd[1]: Created slice GitLab DevOps Platform Slice.758vm-test-run-nixbot-gitlab> gitlab # [    7.173170] systemd[1]: Created slice User and Session Slice.759vm-test-run-nixbot-gitlab> gitlab # [    7.173805] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.760vm-test-run-nixbot-gitlab> gitlab # [    7.174648] systemd[1]: Started Forward Password Requests to Wall Directory Watch.761vm-test-run-nixbot-gitlab> gitlab # [    7.175868] systemd[1]: Expecting device /dev/hvc0...762vm-test-run-nixbot-gitlab> gitlab # [    7.176961] systemd[1]: Expecting device /dev/ttyAMA0...763vm-test-run-nixbot-gitlab> gitlab # [    7.179048] systemd[1]: Reached target Local Encrypted Volumes.764vm-test-run-nixbot-gitlab> gitlab # [    7.180094] systemd[1]: Stopped target initrd-fs.target.765vm-test-run-nixbot-gitlab> gitlab # [    7.180856] systemd[1]: Stopped target initrd-root-fs.target.766vm-test-run-nixbot-gitlab> gitlab # [    7.181555] systemd[1]: Stopped target initrd-switch-root.target.767vm-test-run-nixbot-gitlab> gitlab # [    7.182582] systemd[1]: Reached target Virtual Machines and Containers.768vm-test-run-nixbot-gitlab> gitlab # [    7.183788] systemd[1]: Reached target Path Units.769vm-test-run-nixbot-gitlab> gitlab # [    7.184917] systemd[1]: Reached target Remote File Systems.770vm-test-run-nixbot-gitlab> gitlab # [    7.186034] systemd[1]: Reached target Slice Units.771vm-test-run-nixbot-gitlab> gitlab # [    7.187143] systemd[1]: Reached target Swaps.772vm-test-run-nixbot-gitlab> gitlab # [    7.191660] systemd[1]: Listening on Process Core Dump Socket.773vm-test-run-nixbot-gitlab> gitlab # [    7.194955] systemd[1]: Listening on Credential Encryption/Decryption.774vm-test-run-nixbot-gitlab> gitlab # [    7.203735] systemd[1]: Starting Journal Log Access Socket...775vm-test-run-nixbot-gitlab> gitlab # [    7.206173] systemd[1]: Listening on Journal Audit Socket.776vm-test-run-nixbot-gitlab> gitlab # [    7.207851] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.777vm-test-run-nixbot-gitlab> gitlab # [    7.208701] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki778vm-test-run-nixbot-gitlab> gitlab # [    7.209658] systemd[1]: Listening on udev Control Socket.779vm-test-run-nixbot-gitlab> gitlab # [    7.250307] systemd[1]: Mounting Huge Pages File System...780vm-test-run-nixbot-gitlab> gitlab # [    7.255876] systemd[1]: Mounting POSIX Message Queue File System...781vm-test-run-nixbot-gitlab> gitlab # [    7.262198] systemd[1]: Mounting Kernel Debug File System...782vm-test-run-nixbot-gitlab> gitlab # [    7.270404] systemd[1]: Mounting Kernel Trace File System...783vm-test-run-nixbot-gitlab> gitlab # [    7.276771] systemd[1]: Starting Create List of Static Device Nodes...784vm-test-run-nixbot-gitlab> gitlab # [    7.289006] systemd[1]: Starting Load Kernel Module 9pnet_virtio...785vm-test-run-nixbot-gitlab> gitlab # [    7.291493] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs786vm-test-run-nixbot-gitlab> gitlab # [    7.306027] systemd[1]: Mounting Kernel Configuration File System...787vm-test-run-nixbot-gitlab> gitlab # [    7.306820] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm788vm-test-run-nixbot-gitlab> gitlab # [    7.307635] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore789vm-test-run-nixbot-gitlab> gitlab # [    7.321735] systemd[1]: Starting Load Kernel Module fuse...790vm-test-run-nixbot-gitlab> gitlab # [    7.322639] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67791vm-test-run-nixbot-gitlab> gitlab # [    7.332039] systemd[1]: Starting Journal Service...792vm-test-run-nixbot-gitlab> gitlab # [    7.335696] systemd[1]: Starting Load Kernel Modules...793vm-test-run-nixbot-gitlab> gitlab # [    7.341069] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...794vm-test-run-nixbot-gitlab> gitlab # [    7.350046] systemd[1]: Starting Remount Root and Kernel File Systems...795vm-test-run-nixbot-gitlab> gitlab # [    7.350690] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki796vm-test-run-nixbot-gitlab> gitlab # [    7.361027] systemd[1]: Starting Coldplug All udev Devices...797vm-test-run-nixbot-gitlab> gitlab # [    7.361597] fuse: init (API version 7.45)798vm-test-run-nixbot-gitlab> gitlab # [    7.377133] systemd[1]: Listening on Journal Log Access Socket.799vm-test-run-nixbot-gitlab> gitlab # [    7.378865] systemd[1]: Mounted Huge Pages File System.800vm-test-run-nixbot-gitlab> gitlab # [    7.379554] systemd[1]: Mounted POSIX Message Queue File System.801vm-test-run-nixbot-gitlab> gitlab # [    7.380433] systemd[1]: Mounted Kernel Debug File System.802vm-test-run-nixbot-gitlab> gitlab # [    7.381393] systemd[1]: Mounted Kernel Trace File System.803vm-test-run-nixbot-gitlab> gitlab # [    7.382433] systemd[1]: Finished Create List of Static Device Nodes.804vm-test-run-nixbot-gitlab> gitlab # [    7.387975] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.805vm-test-run-nixbot-gitlab> gitlab # [    7.390117] systemd[1]: Finished Load Kernel Module 9pnet_virtio.806vm-test-run-nixbot-gitlab> gitlab # [    7.391158] systemd[1]: Mounted Kernel Configuration File System.807vm-test-run-nixbot-gitlab> gitlab # [    7.393336] systemd[1]: modprobe@fuse.service: Deactivated successfully.808vm-test-run-nixbot-gitlab> gitlab # [    7.394414] systemd[1]: Finished Load Kernel Module fuse.809vm-test-run-nixbot-gitlab> gitlab # [    7.401736] systemd-journald[787]: Collecting audit messages is enabled.810vm-test-run-nixbot-gitlab> gitlab # [    7.405174] systemd[1]: Mounting FUSE Control File System...811vm-test-run-nixbot-gitlab> gitlab # [    7.406137] EXT4-fs (vda): re-mounted 96be7e04-e448-4d1a-9384-e709bbf319b3.812vm-test-run-nixbot-gitlab> gitlab # [    7.419174] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...813vm-test-run-nixbot-gitlab> gitlab # [    7.425011] systemd[1]: Started Journal Service.814vm-test-run-nixbot-gitlab> gitlab # [    7.419790] systemd[1]: Queued start job for default target Multi-User System.815vm-test-run-nixbot-gitlab> gitlab # [    7.428567] systemd[1]: Unnecessary job was removed for /dev/vdb.816vm-test-run-nixbot-gitlab> gitlab # [    7.429933] systemd[1]: systemd-journald.service: Deactivated successfully.817vm-test-run-nixbot-gitlab> gitlab # [    7.431665] systemd-modules-load[788]: Using 3 probe threads818vm-test-run-nixbot-gitlab> gitlab # [    7.433085] systemd-modules-load[788]: Module 'atkbd' is built in819vm-test-run-nixbot-gitlab> gitlab # [    7.434452] systemd-modules-load[788]: Module 'loop' is built in820vm-test-run-nixbot-gitlab> gitlab # [    7.435760] systemd-modules-load[788]: Inserted module 'tls'821vm-test-run-nixbot-gitlab> gitlab # [    7.448038] systemd-oomd[789]: No swap; memory pressure usage will be degraded822vm-test-run-nixbot-gitlab> gitlab # [    7.456155] systemd[1]: Finished Remount Root and Kernel File Systems.823vm-test-run-nixbot-gitlab> gitlab # [    7.457192] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.824vm-test-run-nixbot-gitlab> gitlab # [    7.458197] systemd[1]: Finished Load Kernel Modules.825vm-test-run-nixbot-gitlab> gitlab # [    7.461932] systemd[1]: Mounted FUSE Control File System.826vm-test-run-nixbot-gitlab> gitlab # [    7.471077] systemd[1]: Starting Firewall...827vm-test-run-nixbot-gitlab> gitlab # [    7.475749] systemd[1]: Starting Flush Journal to Persistent Storage...828vm-test-run-nixbot-gitlab> gitlab # [    7.482164] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore829vm-test-run-nixbot-gitlab> gitlab # [    7.494039] systemd-journald[787]: Received client request to flush runtime journal.830vm-test-run-nixbot-gitlab> gitlab # [    7.541987] systemd[1]: Starting Load/Save OS Random Seed...831vm-test-run-nixbot-gitlab> gitlab # [    7.544656] systemd[1]: Starting Apply Kernel Variables...832vm-test-run-nixbot-gitlab> gitlab # [    7.546278] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki833vm-test-run-nixbot-gitlab> gitlab # [    7.547832] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.834vm-test-run-nixbot-gitlab> gitlab # [    7.549197] systemd[1]: Starting Create Static Device Nodes in /dev...835vm-test-run-nixbot-gitlab> gitlab # [    7.550268] systemd[1]: Finished Load/Save OS Random Seed.836vm-test-run-nixbot-gitlab> gitlab # [    7.551147] systemd[1]: Reached target First Boot Complete.837vm-test-run-nixbot-gitlab> gitlab # [    7.552072] systemd[1]: Finished Apply Kernel Variables.838vm-test-run-nixbot-gitlab> gitlab # [    7.552926] systemd[1]: Finished Create Static Device Nodes in /dev.839vm-test-run-nixbot-gitlab> gitlab # [    7.553906] systemd[1]: Reached target Preparation for Local File Systems.840vm-test-run-nixbot-gitlab> gitlab # [    7.554939] systemd[1]: Starting Rule-based Manager for Device Events and Files...841vm-test-run-nixbot-gitlab> gitlab # [    7.560466] systemd[1]: Finished Flush Journal to Persistent Storage.842vm-test-run-nixbot-gitlab> gitlab # [    7.611818] systemd-udevd[822]: Using default interface naming scheme 'v260'.843vm-test-run-nixbot-gitlab> gitlab # [    7.650877] systemd[1]: Started Rule-based Manager for Device Events and Files.844vm-test-run-nixbot-gitlab> gitlab # [    8.037671] systemd[1]: Finished Firewall.845vm-test-run-nixbot-gitlab> gitlab # [    8.090426] systemd[1]: Finished Coldplug All udev Devices.846vm-test-run-nixbot-gitlab> gitlab # [    8.163252] systemd[1]: Mounting /run/wrappers...847vm-test-run-nixbot-gitlab> gitlab # [    8.193254] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs848vm-test-run-nixbot-gitlab> gitlab # [    8.200856] systemd[1]: Mounted /run/wrappers.849vm-test-run-nixbot-gitlab> gitlab # [    8.205500] systemd[1]: Reached target Local File Systems.850vm-test-run-nixbot-gitlab> gitlab # [    8.208239] systemd[1]: Listening on Boot Loader Control Service Socket.851vm-test-run-nixbot-gitlab> gitlab # [    8.212270] systemd[1]: Starting register-nix-paths.service...852vm-test-run-nixbot-gitlab> gitlab # [    8.220076] systemd[1]: Starting Create SUID/SGID Wrappers...853vm-test-run-nixbot-gitlab> gitlab # [    8.223578] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.854vm-test-run-nixbot-gitlab> gitlab # [    8.248220] systemd[1]: Starting Save Transient machine-id to Disk...855vm-test-run-nixbot-gitlab> gitlab # [    8.252407] systemd[1]: Starting Create System Files and Directories...856vm-test-run-nixbot-gitlab> gitlab # [    8.291563] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.857vm-test-run-nixbot-gitlab> gitlab # [    8.297177] systemd[1]: Finished Save Transient machine-id to Disk.858vm-test-run-nixbot-gitlab> gitlab # [    8.309283] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.859vm-test-run-nixbot-gitlab> gitlab # [    8.340382] systemd[1]: Finished Create System Files and Directories.860vm-test-run-nixbot-gitlab> gitlab # [    8.350740] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.861vm-test-run-nixbot-gitlab> gitlab # [    8.353831] systemd[1]: Starting Rebuild Journal Catalog...862vm-test-run-nixbot-gitlab> gitlab # [    8.355563] systemd[1]: Starting Record System Boot/Shutdown in UTMP...863vm-test-run-nixbot-gitlab> gitlab # [    8.371765] (udev-worker)[946]: Network interface NamePolicy= disabled on kernel command line.864vm-test-run-nixbot-gitlab> gitlab # [    8.391507] (udev-worker)[936]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.865vm-test-run-nixbot-gitlab> gitlab # [    8.397039] (udev-worker)[936]: Network interface NamePolicy= disabled on kernel command line.866vm-test-run-nixbot-gitlab> gitlab # [    8.398615] systemd[1]: Finished Record System Boot/Shutdown in UTMP.867vm-test-run-nixbot-gitlab> gitlab # [    8.413633] systemd[1]: Finished Rebuild Journal Catalog.868vm-test-run-nixbot-gitlab> gitlab # [    8.418960] systemd[1]: Starting Update is Completed...869vm-test-run-nixbot-gitlab> gitlab # [    8.428046] systemd[1]: Condition check resulted in Virtio network device being skipped.870vm-test-run-nixbot-gitlab> gitlab # [    8.439839] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore871vm-test-run-nixbot-gitlab> gitlab # [    8.442058] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.872vm-test-run-nixbot-gitlab> gitlab # [    8.443424] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67873vm-test-run-nixbot-gitlab> gitlab # [    8.446113] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore874vm-test-run-nixbot-gitlab> gitlab # [    8.447839] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki875vm-test-run-nixbot-gitlab> gitlab # [    8.449527] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki876vm-test-run-nixbot-gitlab> gitlab # [    8.451543] systemd[1]: Finished Update is Completed.877vm-test-run-nixbot-gitlab> gitlab # [    8.470489] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.878vm-test-run-nixbot-gitlab> gitlab # [    8.473680] systemd[1]: Finished Create SUID/SGID Wrappers.879vm-test-run-nixbot-gitlab> gitlab # [    8.474489] systemd[1]: suid-sgid-wrappers.service: Consumed 160ms CPU time over 250ms wall clock time, 6.9M memory peak, 2.2M read from disk.880vm-test-run-nixbot-gitlab> gitlab # [    8.518593] mousedev: PS/2 mouse device common for all mice881vm-test-run-nixbot-gitlab> gitlab # [    8.758022] systemd[1]: Finished register-nix-paths.service.882vm-test-run-nixbot-gitlab> gitlab # [    8.758949] systemd[1]: Reached target System Initialization.883vm-test-run-nixbot-gitlab> gitlab # [    8.759730] systemd[1]: Started Discard unused filesystem blocks once a week.884vm-test-run-nixbot-gitlab> gitlab # [    8.760910] systemd[1]: Started logrotate.timer.885vm-test-run-nixbot-gitlab> gitlab # [    8.761619] systemd[1]: Started Daily Cleanup of Temporary Directories.886vm-test-run-nixbot-gitlab> gitlab # [    8.762552] systemd[1]: Reached target Timer Units.887vm-test-run-nixbot-gitlab> gitlab # [    8.763706] systemd[1]: Listening on D-Bus System Message Bus Socket.888vm-test-run-nixbot-gitlab> gitlab # [    8.764808] systemd[1]: Listening on Nix Daemon Socket.889vm-test-run-nixbot-gitlab> gitlab # [    8.773685] systemd[1]: Starting nixbot CI service socket...890vm-test-run-nixbot-gitlab> gitlab # [    8.774692] systemd[1]: Listening on Hostname Service Socket.891vm-test-run-nixbot-gitlab> gitlab # [    8.778681] systemd[1]: Listening on nixbot CI service socket.892vm-test-run-nixbot-gitlab> gitlab # [    8.779677] systemd[1]: Reached target Socket Units.893vm-test-run-nixbot-gitlab> gitlab # [    8.781823] systemd[1]: Starting D-Bus System Message Bus...894vm-test-run-nixbot-gitlab> gitlab # [    8.822239] dbus-broker-launch[1038]: Looking up NSS user entry for 'systemd-timesync'...895vm-test-run-nixbot-gitlab> gitlab # [    8.824406] dbus-broker-launch[1038]: NSS returned no entry for 'systemd-timesync'896vm-test-run-nixbot-gitlab> gitlab # [    8.825579] dbus-broker-launch[1038]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"897vm-test-run-nixbot-gitlab> gitlab # [    8.828898] systemd[1]: Started D-Bus System Message Bus.898vm-test-run-nixbot-gitlab> gitlab # [    8.840866] dbus-broker-launch[1038]: Ready899vm-test-run-nixbot-gitlab> gitlab # [    8.842040] systemd[1]: Reached target Basic System.900vm-test-run-nixbot-gitlab> gitlab # [    8.843590] systemd[1]: Started backdoor.service.901vm-test-run-nixbot-gitlab> gitlab # [    8.848377] systemd[1]: Starting gitlab-config.service...902vm-test-run-nixbot-gitlab> gitlab # [    8.850123] systemd[1]: Starting Import lastlog data into lastlog2 database...903vm-test-run-nixbot-gitlab> gitlab # [    8.855199] systemd[1]: Starting Logrotate configuration check...904vm-test-run-nixbot-gitlab> gitlab # [    8.872293] systemd[1]: Starting Name Service Cache Daemon (nsncd)...905vm-test-run-nixbot-gitlab> gitlab # [    8.879038] systemd[1]: Starting Post-Boot Actions...906vm-test-run-nixbot-gitlab> gitlab # [    8.880202] logrotate[1043]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.907vm-test-run-nixbot-gitlab> gitlab # [    8.883741] logrotate[1043]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf908vm-test-run-nixbot-gitlab> gitlab # [    8.887141] logrotate[1043]: note: 'monthly' overrides previously specified 'weekly'909vm-test-run-nixbot-gitlab> gitlab # connecting to host...910vm-test-run-nixbot-gitlab> gitlab # [    8.888292] logrotate[1043]: Reading state from file: /var/lib/logrotate.status911vm-test-run-nixbot-gitlab> gitlab # [    8.889486] logrotate[1043]: state file /var/lib/logrotate.status does not exist912vm-test-run-nixbot-gitlab> gitlab # [    8.892934] logrotate[1043]: Allocating hash table for state file, size 64 entries913vm-test-run-nixbot-gitlab> gitlab # [    8.893925] logrotate[1043]: Handling 4 logs914vm-test-run-nixbot-gitlab> gitlab # [    8.897971] logrotate[1043]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed915vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'916vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell917vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.33 seconds)918vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.33 seconds)919vm-test-run-nixbot-gitlab> gitlab # [    8.902550] logrotate[1043]: considering log /var/log/btmp920vm-test-run-nixbot-gitlab> gitlab # [    8.903234] logrotate[1043]: Creating new state921vm-test-run-nixbot-gitlab> gitlab # [    8.903806] logrotate[1043]:   Now: 2026-06-14 14:25922vm-test-run-nixbot-gitlab> gitlab # [    8.904550] logrotate[1043]:   Last rotated at 2026-06-14 14:00923vm-test-run-nixbot-gitlab> gitlab # [    8.905249] logrotate[1043]:   log does not need rotating (log has already been rotated)924vm-test-run-nixbot-gitlab> gitlab # [    8.906178] logrotate[1043]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed925vm-test-run-nixbot-gitlab> gitlab # [    8.909049] logrotate[1043]: considering log /var/log/wtmp926vm-test-run-nixbot-gitlab> gitlab # [    8.909724] logrotate[1043]: Creating new state927vm-test-run-nixbot-gitlab> gitlab # [    8.910254] logrotate[1043]:   Now: 2026-06-14 14:25928vm-test-run-nixbot-gitlab> gitlab # [    8.910825] logrotate[1043]:   Last rotated at 2026-06-14 14:00929vm-test-run-nixbot-gitlab> gitlab # [    8.911549] logrotate[1043]:   log does not need rotating (log has already been rotated)930vm-test-run-nixbot-gitlab> gitlab # [    8.912713] logrotate[1043]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed931vm-test-run-nixbot-gitlab> gitlab # [    8.914517] logrotate[1043]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1043)932vm-test-run-nixbot-gitlab> gitlab # [    8.918403] logrotate[1043]: considering log /var/gitlab/state/log/*.log933vm-test-run-nixbot-gitlab> gitlab # [    8.919295] logrotate[1043]:   log /var/gitlab/state/log/*.log does not exist -- skipping934vm-test-run-nixbot-gitlab> gitlab # [    8.920364] logrotate[1043]: Creating new state935vm-test-run-nixbot-gitlab> gitlab # [    8.920959] logrotate[1043]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1043)936vm-test-run-nixbot-gitlab> gitlab # [    8.921984] logrotate[1043]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed937vm-test-run-nixbot-gitlab> gitlab # [    8.925971] logrotate[1043]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1043)938vm-test-run-nixbot-gitlab> gitlab # [    8.926995] logrotate[1043]: considering log /var/log/nginx/*.log939vm-test-run-nixbot-gitlab> gitlab # [    8.927755] logrotate[1043]:   log /var/log/nginx/*.log does not exist -- skipping940vm-test-run-nixbot-gitlab> gitlab # [    8.928808] logrotate[1043]: Creating new state941vm-test-run-nixbot-gitlab> gitlab # [    8.929387] logrotate[1043]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1043)942vm-test-run-nixbot-gitlab> gitlab # [    8.930517] systemd[1]: Started Reset console on configuration changes.943vm-test-run-nixbot-gitlab> gitlab # [    8.931742] nsncd[1044]: Jun 14 14:25:01.825 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"944vm-test-run-nixbot-gitlab> gitlab # [    8.933888] systemd[1]: Starting resolvconf update...945vm-test-run-nixbot-gitlab> gitlab # [    8.934728] systemd[1]: Finished Logrotate configuration check.946vm-test-run-nixbot-gitlab> gitlab # [    8.935669] systemd[1]: Finished Post-Boot Actions.947vm-test-run-nixbot-gitlab> gitlab # [    8.936935] systemd[1]: Finished Import lastlog data into lastlog2 database.948vm-test-run-nixbot-gitlab> gitlab # [    8.938040] systemd[1]: Started Name Service Cache Daemon (nsncd).949vm-test-run-nixbot-gitlab> gitlab # [    8.939024] systemd[1]: Reached target Host and Network Name Lookups.950vm-test-run-nixbot-gitlab> gitlab # [    8.940456] systemd[1]: Reached target User and Group Name Lookups.951vm-test-run-nixbot-gitlab> gitlab # [    8.941276] systemd[1]: Starting User Login Management...952vm-test-run-nixbot-gitlab> gitlab # [    9.028099] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input3953vm-test-run-nixbot-gitlab> gitlab # [    9.038823] systemd[1]: Stopped target Host and Network Name Lookups.954vm-test-run-nixbot-gitlab> gitlab # [    9.040455] systemd[1]: Stopping Host and Network Name Lookups...955vm-test-run-nixbot-gitlab> gitlab # [    9.041288] systemd[1]: Stopped target User and Group Name Lookups.956vm-test-run-nixbot-gitlab> gitlab # [    9.042171] systemd[1]: Stopping User and Group Name Lookups...957vm-test-run-nixbot-gitlab> gitlab # [    9.044697] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...958vm-test-run-nixbot-gitlab> gitlab # [    9.050227] systemd[1]: nscd.service: Deactivated successfully.959vm-test-run-nixbot-gitlab> gitlab # [    9.051090] systemd[1]: Stopped Name Service Cache Daemon (nsncd).960vm-test-run-nixbot-gitlab> gitlab # [    9.051945] systemd[1]: nscd.service: Consumed 25ms CPU time over 176ms wall clock time, 8.2M memory peak, 5.9M read from disk.961vm-test-run-nixbot-gitlab> gitlab # [    9.060673] systemd-logind[1075]: New seat seat0.962vm-test-run-nixbot-gitlab> gitlab # [    9.064929] systemd[1]: Starting Name Service Cache Daemon (nsncd)...963vm-test-run-nixbot-gitlab> gitlab # [    9.065827] systemd-logind[1075]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)964vm-test-run-nixbot-gitlab> gitlab # [    9.067272] systemd-logind[1075]: Watching system buttons on /dev/input/event0 (gpio-keys)965vm-test-run-nixbot-gitlab> gitlab # [    9.068987] systemd[1]: Started User Login Management.966vm-test-run-nixbot-gitlab> gitlab # [    9.083514] systemd[1]: Starting linger-users.service...967vm-test-run-nixbot-gitlab> gitlab # [    9.097020] nsncd[1134]: Jun 14 14:25:01.997 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"968vm-test-run-nixbot-gitlab> gitlab # [    9.099106] systemd[1]: Started Name Service Cache Daemon (nsncd).969vm-test-run-nixbot-gitlab> gitlab # [    9.102181] systemd[1]: Finished resolvconf update.970vm-test-run-nixbot-gitlab> gitlab # [    9.102990] systemd[1]: Reached target Preparation for Network.971vm-test-run-nixbot-gitlab> gitlab # [    9.103772] systemd[1]: Reached target Host and Network Name Lookups.972vm-test-run-nixbot-gitlab> gitlab # [    9.104845] systemd[1]: Reached target User and Group Name Lookups.973vm-test-run-nixbot-gitlab> gitlab # [    9.109002] systemd[1]: Starting DHCP Client...974vm-test-run-nixbot-gitlab> gitlab # [    9.117767] systemd[1]: Starting Address configuration of eth1...975vm-test-run-nixbot-gitlab> gitlab # [    9.128088] systemd[1]: Starting Extra networking commands....976vm-test-run-nixbot-gitlab> gitlab # [    9.128938] systemd[1]: linger-users.service: Deactivated successfully.977vm-test-run-nixbot-gitlab> gitlab # [    9.136180] systemd[1]: Finished linger-users.service.978vm-test-run-nixbot-gitlab> gitlab # [    9.163034] network-addresses-eth1-start[1157]: adding address 192.168.1.1/24... done979vm-test-run-nixbot-gitlab> gitlab # [    9.168643] systemd[1]: Starting Virtual Console Setup...980vm-test-run-nixbot-gitlab> gitlab # [    9.187150] network-addresses-eth1-start[1157]: adding address 2001:db8:1::1/64... done981vm-test-run-nixbot-gitlab> gitlab # [    9.207454] systemd[1]: Finished Address configuration of eth1.982vm-test-run-nixbot-gitlab> gitlab # [    9.218011] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.983vm-test-run-nixbot-gitlab> gitlab # [    9.219173] systemd[1]: Stopped Virtual Console Setup.984vm-test-run-nixbot-gitlab> gitlab # [    9.224756] systemd[1]: Starting Virtual Console Setup...985vm-test-run-nixbot-gitlab> gitlab # [    9.238563] systemd-logind[1075]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)986vm-test-run-nixbot-gitlab> gitlab # [    9.245791] systemd[1]: Finished Extra networking commands..987vm-test-run-nixbot-gitlab> gitlab # [    9.247671] systemd[1]: Reached target Network.988vm-test-run-nixbot-gitlab> gitlab # [    9.252068] systemd[1]: Starting gitlab-workhorse.service...989vm-test-run-nixbot-gitlab> gitlab # [    9.258937] systemd[1]: Starting Nginx Web Server...990vm-test-run-nixbot-gitlab> gitlab # [    9.260987] systemd[1]: Starting PostgreSQL Server...991vm-test-run-nixbot-gitlab> gitlab # [    9.269427] systemd[1]: Starting Redis Server - redis-gitlab...992vm-test-run-nixbot-gitlab> gitlab # [    9.276553] systemd[1]: Starting Permit User Sessions...993vm-test-run-nixbot-gitlab> gitlab # [    9.282702] dhcpcd[1183]: dhcpcd-10.3.2 starting994vm-test-run-nixbot-gitlab> gitlab # [    9.286659] dhcpcd[1229]: dev: loaded udev995vm-test-run-nixbot-gitlab> gitlab # [    9.299645] 8021q: 802.1Q VLAN Support v1.8996vm-test-run-nixbot-gitlab> gitlab # [    9.300017] 8021q: adding VLAN 0 to HW filter on device eth1997vm-test-run-nixbot-gitlab> gitlab # [    9.308888] systemd-vconsole-setup[1208]: Configuration of first virtual console was skipped, ignoring remaining ones.998vm-test-run-nixbot-gitlab> gitlab # [    9.312256] systemd[1]: Finished Virtual Console Setup.999vm-test-run-nixbot-gitlab> gitlab # [    9.317909] systemd[1]: Finished Permit User Sessions.1000vm-test-run-nixbot-gitlab> gitlab # [    9.327833] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1001vm-test-run-nixbot-gitlab> gitlab # [    9.332158] systemd[1]: Started Getty on tty1.1002vm-test-run-nixbot-gitlab> gitlab # [    9.332840] systemd[1]: Reached target Login Prompts.1003vm-test-run-nixbot-gitlab> gitlab # [    9.355705] cfg80211: Loading compiled-in X.509 certificates for regulatory database1004vm-test-run-nixbot-gitlab> gitlab # [    9.369605] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1005vm-test-run-nixbot-gitlab> gitlab # [    9.370239] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1006vm-test-run-nixbot-gitlab> gitlab # [    9.371973] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21007vm-test-run-nixbot-gitlab> gitlab # [    9.372315] cfg80211: failed to load regulatory.db1008vm-test-run-nixbot-gitlab> gitlab # [    9.418172] dhcpcd[1229]: eth0: waiting for carrier1009vm-test-run-nixbot-gitlab> gitlab # [    9.419715] dhcpcd[1229]: eth0: carrier acquired[    9.418188] 8021q: adding VLAN 0 to HW filter on device eth01010vm-test-run-nixbot-gitlab> gitlab # 1011vm-test-run-nixbot-gitlab> gitlab # [    9.426024] dhcpcd[1229]: DUID 00:01:00:01:31:c1:76:3e:52:54:00:12:34:561012vm-test-run-nixbot-gitlab> gitlab # [    9.427038] dhcpcd[1229]: eth0: IAID 00:12:34:561013vm-test-run-nixbot-gitlab> gitlab # [    9.427806] dhcpcd[1229]: eth0: adding address fe80::5054:ff:fe12:34561014vm-test-run-nixbot-gitlab> gitlab # [    9.452963] nginx-pre-start[1251]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok1015vm-test-run-nixbot-gitlab> gitlab # [    9.456538] nginx-pre-start[1251]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful1016vm-test-run-nixbot-gitlab> gitlab # [    9.459192] redis-server[1250]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1017vm-test-run-nixbot-gitlab> gitlab # [    9.462436] redis-server[1250]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1018vm-test-run-nixbot-gitlab> gitlab # [    9.465951] postgresql-pre-start[1256]: The files belonging to this database system will be owned by user "postgres".1019vm-test-run-nixbot-gitlab> gitlab # [    9.467298] postgresql-pre-start[1256]: This user must also own the server process.1020vm-test-run-nixbot-gitlab> gitlab # [    9.471749] postgresql-pre-start[1256]: The database cluster will be initialized with locale "en_US.UTF-8".1021vm-test-run-nixbot-gitlab> gitlab # [    9.473194] postgresql-pre-start[1256]: The default database encoding has accordingly been set to "UTF8".1022vm-test-run-nixbot-gitlab> gitlab # [    9.475242] postgresql-pre-start[1256]: The default text search configuration will be set to "english".1023vm-test-run-nixbot-gitlab> gitlab # [    9.477616] postgresql-pre-start[1256]: Data page checksums are disabled.1024vm-test-run-nixbot-gitlab> gitlab # [    9.479088] postgresql-pre-start[1256]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1025vm-test-run-nixbot-gitlab> gitlab # [    9.482478] systemd[1]: Started Nginx Web Server.1026vm-test-run-nixbot-gitlab> gitlab # [    9.483151] redis-server[1250]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1250, just started1027vm-test-run-nixbot-gitlab> gitlab # [    9.484829] postgresql-pre-start[1256]: creating subdirectories ... ok1028vm-test-run-nixbot-gitlab> gitlab # [    9.485963] postgresql-pre-start[1256]: selecting dynamic shared memory implementation ... posix1029vm-test-run-nixbot-gitlab> gitlab # [    9.489587] systemd[1]: Started Redis Server - redis-gitlab.1030vm-test-run-nixbot-gitlab> gitlab # [    9.490614] redis-server[1250]: Configuration loaded1031vm-test-run-nixbot-gitlab> gitlab # [    9.492226] redis[1250]: monotonic clock: ARM CNTVCT @ 1000 ticks/us1032vm-test-run-nixbot-gitlab> gitlab # [    9.493381] redis[1250]: Running mode=standalone, port=0.1033vm-test-run-nixbot-gitlab> gitlab # [    9.494311] redis[1250]: Server initialized1034vm-test-run-nixbot-gitlab> gitlab # [    9.495053] redis[1250]: Ready to accept connections unix1035vm-test-run-nixbot-gitlab> gitlab # [    9.495949] redis[1250]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1036vm-test-run-nixbot-gitlab> gitlab # [    9.509286] postgresql-pre-start[1256]: selecting default "max_connections" ... 1001037vm-test-run-nixbot-gitlab> gitlab # [    9.560767] postgresql-pre-start[1256]: selecting default "shared_buffers" ... 128MB1038vm-test-run-nixbot-gitlab> gitlab # [    9.767276] systemd[1]: Started gitlab-workhorse.service.1039vm-test-run-nixbot-gitlab> gitlab # [    9.978992] postgresql-pre-start[1256]: selecting default time zone ... UTC1040vm-test-run-nixbot-gitlab> gitlab # [    9.982564] postgresql-pre-start[1256]: creating configuration files ... ok1041vm-test-run-nixbot-gitlab> gitlab # [   10.176589] gitlab-workhorse[1286]: time="2026-06-14T14:25:03Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41042vm-test-run-nixbot-gitlab> gitlab # [   10.178288] gitlab-workhorse[1286]: time="2026-06-14T14:25:03Z" level=info msg="Using redis/go-redis"1043vm-test-run-nixbot-gitlab> gitlab # [   10.179445] gitlab-workhorse[1286]: time="2026-06-14T14:25:03Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1044vm-test-run-nixbot-gitlab> gitlab # [   10.247339] postgresql-pre-start[1256]: running bootstrap script ... ok1045vm-test-run-nixbot-gitlab> gitlab # [   11.028480] postgresql-pre-start[1256]: performing post-bootstrap initialization ... ok1046vm-test-run-nixbot-gitlab> gitlab # [   11.120683] dhcpcd[1229]: eth0: soliciting a DHCP lease1047vm-test-run-nixbot-gitlab> gitlab # [   11.140691] dhcpcd[1229]: eth0: offered 10.0.2.15 from 10.0.2.21048vm-test-run-nixbot-gitlab> gitlab # [   11.156393] dhcpcd[1229]: eth0: probing address 10.0.2.15/241049vm-test-run-nixbot-gitlab> gitlab # [   11.298885] postgresql-pre-start[1256]: syncing data to disk ... ok1050vm-test-run-nixbot-gitlab> gitlab # [   11.301592] postgresql-pre-start[1256]: initdb: warning: enabling "trust" authentication for local connections1051vm-test-run-nixbot-gitlab> gitlab # [   11.305474] postgresql-pre-start[1256]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1052vm-test-run-nixbot-gitlab> gitlab # [   11.311573] postgresql-pre-start[1256]: Success. You can now start the database server using:1053vm-test-run-nixbot-gitlab> gitlab # [   11.315070] postgresql-pre-start[1256]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1054vm-test-run-nixbot-gitlab> gitlab # [   11.486230] postgres[1320]: [1320] LOG:  starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit1055vm-test-run-nixbot-gitlab> gitlab # [   11.492255] postgres[1320]: [1320] LOG:  listening on IPv6 address "::1", port 54321056vm-test-run-nixbot-gitlab> gitlab # [   11.495095] postgres[1320]: [1320] LOG:  listening on IPv4 address "127.0.0.1", port 54321057vm-test-run-nixbot-gitlab> gitlab # [   11.499061] postgres[1320]: [1320] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1058vm-test-run-nixbot-gitlab> gitlab # [   11.503324] postgres[1330]: [1330] LOG:  database system was shut down at 2026-06-14 14:25:03 GMT1059vm-test-run-nixbot-gitlab> gitlab # [   11.506952] postgres[1320]: [1320] LOG:  database system is ready to accept connections1060vm-test-run-nixbot-gitlab> gitlab # [   11.511718] systemd[1]: Started PostgreSQL Server.1061vm-test-run-nixbot-gitlab> gitlab # [   11.515922] systemd[1]: Starting PostgreSQL Setup Scripts...1062vm-test-run-nixbot-gitlab> gitlab # [   11.686546] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1327]: mkdir -p /var/gitlab/state/home/.ssh: OK1063vm-test-run-nixbot-gitlab> gitlab # [   11.691684] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1327]: chmod 700 /var/gitlab/state/home/.ssh: OK1064vm-test-run-nixbot-gitlab> gitlab # [   11.763679] systemd[1]: Finished gitlab-config.service.1065vm-test-run-nixbot-gitlab> gitlab # [   11.772620] systemd[1]: Started gitaly.service.1066vm-test-run-nixbot-gitlab> gitlab # [   11.805242] postgresql-setup-start[1342]: CREATE DATABASE1067vm-test-run-nixbot-gitlab> gitlab # [   11.862118] postgresql-setup-start[1361]: CREATE ROLE1068vm-test-run-nixbot-gitlab> gitlab # [   11.887357] postgresql-setup-start[1363]: ALTER DATABASE1069vm-test-run-nixbot-gitlab> gitlab # [   11.940695] postgresql-setup-start[1368]: CREATE ROLE1070vm-test-run-nixbot-gitlab> gitlab # [   11.943717] systemd[1]: Finished PostgreSQL Setup Scripts.1071vm-test-run-nixbot-gitlab> gitlab # [   11.946367] systemd[1]: Reached target PostgreSQL.1072vm-test-run-nixbot-gitlab> gitlab # [   11.952129] systemd[1]: Starting gitlab-db-config.service...1073vm-test-run-nixbot-gitlab> gitlab # [   11.957153] systemd[1]: Starting gitlab-postgresql.service...1074vm-test-run-nixbot-gitlab> gitlab # [   12.154827] gitlab-postgresql-start[1379]: CREATE DATABASE1075vm-test-run-nixbot-gitlab> gitlab # [   12.200702] gitlab-postgresql-start[1396]: CREATE EXTENSION1076vm-test-run-nixbot-gitlab> gitlab # [   12.248666] gitlab-postgresql-start[1400]: CREATE EXTENSION1077vm-test-run-nixbot-gitlab> gitlab # [   12.252390] systemd[1]: Finished gitlab-postgresql.service.1078vm-test-run-nixbot-gitlab> gitlab # [   12.348465] dhcpcd[1229]: eth0: soliciting an IPv6 router1079vm-test-run-nixbot-gitlab> gitlab # [   12.351076] dhcpcd[1229]: eth0: Router Advertisement from fe80::21080vm-test-run-nixbot-gitlab> gitlab # [   12.354000] dhcpcd[1229]: eth0: adding address fec0::5054:ff:fe12:3456/641081vm-test-run-nixbot-gitlab> gitlab # [   12.356737] dhcpcd[1229]: eth0: adding route to fec0::/641082vm-test-run-nixbot-gitlab> gitlab # [   12.359193] dhcpcd[1229]: eth0: adding default route via fe80::21083vm-test-run-nixbot-gitlab> gitlab # [   12.495992] gitaly[1356]: time="2026-06-14T14:25:05.395Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=13561084vm-test-run-nixbot-gitlab> gitlab # [   12.510348] gitaly[1356]: time="2026-06-14T14:25:05.396Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13561085vm-test-run-nixbot-gitlab> gitlab # [   12.511825] gitaly[1356]: time="2026-06-14T14:25:05.396Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13561086vm-test-run-nixbot-gitlab> gitlab # [   12.514111] gitaly[1356]: time="2026-06-14T14:25:05.396Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13561087vm-test-run-nixbot-gitlab> gitlab # [   12.517337] gitaly[1356]: time="2026-06-14T14:25:05.396Z" level=info msg="Starting Gitaly" pid=1356 version=18.11.41088vm-test-run-nixbot-gitlab> gitlab # [   12.766880] gitaly[1356]: time="2026-06-14T14:25:05.667Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=270 pid=13561089vm-test-run-nixbot-gitlab> gitlab # [   12.769337] gitaly[1356]: time="2026-06-14T14:25:05.669Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13561090vm-test-run-nixbot-gitlab> gitlab # [   12.770736] gitaly[1356]: time="2026-06-14T14:25:05.669Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13561091vm-test-run-nixbot-gitlab> gitlab # [   12.772331] gitaly[1356]: time="2026-06-14T14:25:05.669Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1356/git-exec-3795578791.d/git pid=13561092vm-test-run-nixbot-gitlab> gitlab # [   12.784112] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="finished detecting git version" duration_ms=11 pid=13561093vm-test-run-nixbot-gitlab> gitlab # [   12.785636] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="using Git version" pid=1356 version=2.53.g1094vm-test-run-nixbot-gitlab> gitlab # [   12.786857] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="clearing disk cache object folder" pid=1356 storage=default1095vm-test-run-nixbot-gitlab> gitlab # [   12.788292] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1937097514 pid=1356 storage=default1096vm-test-run-nixbot-gitlab> gitlab # [   12.791270] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1937097514 pid=1356 storage=default1097vm-test-run-nixbot-gitlab> gitlab # [   12.795102] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13561098vm-test-run-nixbot-gitlab> gitlab # [   12.798342] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13561099vm-test-run-nixbot-gitlab> gitlab # [   12.800986] gitaly[1356]: time="2026-06-14T14:25:05.681Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1937097514 clear_duration_ms=0 pid=1356 storage=default1100vm-test-run-nixbot-gitlab> gitlab # [   12.806120] gitaly[1356]: time="2026-06-14T14:25:05.706Z" level=info msg="counting repositories" pid=13561101vm-test-run-nixbot-gitlab> gitlab # [   12.807509] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="starting to count repositories" pid=1356 storage_path=/var/gitlab/state/repositories1102vm-test-run-nixbot-gitlab> gitlab # [   12.809966] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1356 storage_path=/var/gitlab/state/repositories1103vm-test-run-nixbot-gitlab> gitlab # [   12.812949] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13561104vm-test-run-nixbot-gitlab> gitlab # [   12.815036] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="finished tempdir cleaner walk" pid=1356 storage=default time_ms=01105vm-test-run-nixbot-gitlab> gitlab # [   12.817392] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13561106vm-test-run-nixbot-gitlab> gitlab # [   12.820089] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1356 schema=unix1107vm-test-run-nixbot-gitlab> gitlab # [   12.822461] gitaly[1356]: time="2026-06-14T14:25:05.707Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1356/sock.d/intern pid=1356 schema=unix1108vm-test-run-nixbot-gitlab> gitlab # [   12.824753] gitaly[1356]: time="2026-06-14T14:25:05.708Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13561109vm-test-run-nixbot-gitlab> gitlab # [   12.826784] gitaly[1356]: time="2026-06-14T14:25:05.708Z" level=info msg="Started Gitaly" duration_ms=312 pid=13561110vm-test-run-nixbot-gitlab> gitlab # [   12.828472] gitaly[1356]: time="2026-06-14T14:25:05.709Z" level=info msg="maintenance: daily scheduled" pid=1356 scheduled="2026-06-15 12:00:00 +0000 UTC"1111vm-test-run-nixbot-gitlab> gitlab # [   13.232992] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1112vm-test-run-nixbot-gitlab> gitlab # [   13.241122] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1113vm-test-run-nixbot-gitlab> gitlab # [   13.253735] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1114vm-test-run-nixbot-gitlab> gitlab # [   15.557172] dhcpcd[1229]: eth0: leased 10.0.2.15 for 86400 seconds1115vm-test-run-nixbot-gitlab> gitlab # [   15.559864] dhcpcd[1229]: eth0: adding route to 10.0.2.0/241116vm-test-run-nixbot-gitlab> gitlab # [   15.562274] dhcpcd[1229]: eth0: adding default route via 10.0.2.21117vm-test-run-nixbot-gitlab> gitlab # [   15.727228] systemd[1]: Started DHCP Client.1118vm-test-run-nixbot-gitlab> gitlab # [   15.732785] systemd[1]: Reached target Network is Online.1119vm-test-run-nixbot-gitlab> gitlab # [   16.061103] gitaly[1356]: time="2026-06-14T14:25:08.960Z" level=info msg="License database preloaded" duration_ms=3564 pid=13561120vm-test-run-nixbot-gitlab> gitlab # [   55.669934] postgres[1836]: [1836] ERROR:  relation "feature_gates" does not exist at character 5231121vm-test-run-nixbot-gitlab> gitlab # [   55.670613] postgres[1836]: [1836] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1122vm-test-run-nixbot-gitlab> gitlab # [   55.670779] postgres[1836]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1123vm-test-run-nixbot-gitlab> gitlab # [   55.670898] postgres[1836]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1124vm-test-run-nixbot-gitlab> gitlab # [   55.671016] postgres[1836]: 	       attidentity AS identity,1125vm-test-run-nixbot-gitlab> gitlab # [   55.671132] postgres[1836]: 	       attgenerated as attgenerated1126vm-test-run-nixbot-gitlab> gitlab # [   55.671248] postgres[1836]: 	  FROM pg_attribute a1127vm-test-run-nixbot-gitlab> gitlab # [   55.671349] postgres[1836]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1128vm-test-run-nixbot-gitlab> gitlab # [   55.671465] postgres[1836]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1129vm-test-run-nixbot-gitlab> gitlab # [   55.671600] postgres[1836]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1130vm-test-run-nixbot-gitlab> gitlab # [   55.671717] postgres[1836]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1131vm-test-run-nixbot-gitlab> gitlab # [   55.671832] postgres[1836]: 	   AND a.attnum > 0 AND NOT a.attisdropped1132vm-test-run-nixbot-gitlab> gitlab # [   55.671933] postgres[1836]: 	 ORDER BY a.attnum1133vm-test-run-nixbot-gitlab> gitlab # [   55.702272] postgres[1836]:1134vm-test-run-nixbot-gitlab> gitlab # [   55.979060] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1375]: W, [2026-06-14T14:25:48.878659 #1375]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1135vm-test-run-nixbot-gitlab> gitlab # [   60.249294] gitlab-workhorse[1286]: time="2026-06-14T14:25:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1136vm-test-run-nixbot-gitlab> gitlab # [   63.646970] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1918]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1137vm-test-run-nixbot-gitlab> gitlab # [   63.647366] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1918]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1138vm-test-run-nixbot-gitlab> gitlab # [   70.257790] gitlab-workhorse[1286]: time="2026-06-14T14:26:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1139vm-test-run-nixbot-gitlab> gitlab # [   80.268111] gitlab-workhorse[1286]: time="2026-06-14T14:26:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1140vm-test-run-nixbot-gitlab> gitlab # [   87.306195] postgres[2003]: [2003] ERROR:  relation "table_without_models" does not exist1141vm-test-run-nixbot-gitlab> gitlab # [   87.308360] postgres[2003]: [2003] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1142vm-test-run-nixbot-gitlab> gitlab # [   90.278267] gitlab-workhorse[1286]: time="2026-06-14T14:26:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143vm-test-run-nixbot-gitlab> gitlab # [  100.286492] gitlab-workhorse[1286]: time="2026-06-14T14:26:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144vm-test-run-nixbot-gitlab> gitlab # [  110.296777] gitlab-workhorse[1286]: time="2026-06-14T14:26:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145vm-test-run-nixbot-gitlab> gitlab # [  117.030533] systemd[1]: Finished gitlab-db-config.service.1146vm-test-run-nixbot-gitlab> gitlab # [  117.036911] systemd[1]: Started gitlab-sidekiq.service.1147vm-test-run-nixbot-gitlab> gitlab # [  117.041831] systemd[1]: Starting gitlab.service...1148vm-test-run-nixbot-gitlab> gitlab # [  117.564705] bundle[2369]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1149vm-test-run-nixbot-gitlab> gitlab # [  117.570694] bundle[2369]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1150vm-test-run-nixbot-gitlab> gitlab # [  117.576354] bundle[2369]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1151vm-test-run-nixbot-gitlab> gitlab # [  120.297789] gitlab-workhorse[1286]: time="2026-06-14T14:26:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1152vm-test-run-nixbot-gitlab> gitlab # [  122.033999] sidekiq-cluster[2368]: {"severity":"INFO","time":"2026-06-14T14:26:54.907Z","message":"Starting cluster with 1 processes"}1153vm-test-run-nixbot-gitlab> gitlab # [  122.125450] bundle[2369]: {"timestamp":"2026-06-14T14:26:54.998Z","pid":2369,"message":"* Enabling systemd notification integration"}1154vm-test-run-nixbot-gitlab> gitlab # [  122.130166] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"Puma starting in cluster mode..."}1155vm-test-run-nixbot-gitlab> gitlab # [  122.134388] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1156vm-test-run-nixbot-gitlab> gitlab # [  122.139839] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1157vm-test-run-nixbot-gitlab> gitlab # [  122.145448] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*  Min threads: 0"}1158vm-test-run-nixbot-gitlab> gitlab # [  122.149128] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*  Max threads: 4"}1159vm-test-run-nixbot-gitlab> gitlab # [  122.152898] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*  Environment: production"}1160vm-test-run-nixbot-gitlab> gitlab # [  122.157026] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*   Master PID: 2369"}1161vm-test-run-nixbot-gitlab> gitlab # [  122.161156] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*      Workers: 2"}1162vm-test-run-nixbot-gitlab> gitlab # [  122.164995] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1163vm-test-run-nixbot-gitlab> gitlab # [  122.170178] bundle[2369]: {"timestamp":"2026-06-14T14:26:55.025Z","pid":2369,"message":"* Preloading application"}1164vm-test-run-nixbot-gitlab> gitlab # [  130.301985] gitlab-workhorse[1286]: time="2026-06-14T14:27:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1165vm-test-run-nixbot-gitlab> gitlab # [  140.303384] gitlab-workhorse[1286]: time="2026-06-14T14:27:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1166vm-test-run-nixbot-gitlab> gitlab # [  150.304581] gitlab-workhorse[1286]: time="2026-06-14T14:27:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1167vm-test-run-nixbot-gitlab> gitlab # [  160.272906] bundle[2369]: W, [2026-06-14T14:27:33.172353 #2369]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1168vm-test-run-nixbot-gitlab> gitlab # [  160.314411] gitlab-workhorse[1286]: time="2026-06-14T14:27:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1169vm-test-run-nixbot-gitlab> gitlab # [  165.468987] sidekiq-cluster[2418]: 2026-06-14T14:27:38.368Z pid=2418 tid=ttm INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2418, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>451543040}1170vm-test-run-nixbot-gitlab> gitlab # [  165.763145] sidekiq-cluster[2418]: W, [2026-06-14T14:27:38.663711 #2418]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1171vm-test-run-nixbot-gitlab> gitlab # [  165.823307] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:38.723Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1172vm-test-run-nixbot-gitlab> gitlab # [  168.899059] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.798Z","message":"GitLab reliable fetch activated!"}1173vm-test-run-nixbot-gitlab> gitlab # [  168.899375] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.798Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1174vm-test-run-nixbot-gitlab> gitlab # [  168.933884] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.834Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1175vm-test-run-nixbot-gitlab> gitlab # [  168.940430] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.837Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1176vm-test-run-nixbot-gitlab> gitlab # [  168.946543] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.840Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1177vm-test-run-nixbot-gitlab> gitlab # [  168.953606] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.842Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1178vm-test-run-nixbot-gitlab> gitlab # [  168.959935] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.845Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1179vm-test-run-nixbot-gitlab> gitlab # [  168.966517] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.847Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1180vm-test-run-nixbot-gitlab> gitlab # [  168.972419] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.849Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1181vm-test-run-nixbot-gitlab> gitlab # [  168.977585] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.851Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1182vm-test-run-nixbot-gitlab> gitlab # [  168.982400] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.853Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1183vm-test-run-nixbot-gitlab> gitlab # [  168.986988] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.855Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1184vm-test-run-nixbot-gitlab> gitlab # [  168.991698] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.857Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1185vm-test-run-nixbot-gitlab> gitlab # [  168.995911] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.860Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1186vm-test-run-nixbot-gitlab> gitlab # [  168.999598] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.862Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1187vm-test-run-nixbot-gitlab> gitlab # [  169.003332] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.864Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1188vm-test-run-nixbot-gitlab> gitlab # [  169.006802] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.866Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1189vm-test-run-nixbot-gitlab> gitlab # [  169.010127] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.869Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1190vm-test-run-nixbot-gitlab> gitlab # [  169.013311] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.871Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1191vm-test-run-nixbot-gitlab> gitlab # [  169.016089] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.873Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1192vm-test-run-nixbot-gitlab> gitlab # [  169.018698] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.876Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1193vm-test-run-nixbot-gitlab> gitlab # [  169.021663] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.879Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1194vm-test-run-nixbot-gitlab> gitlab # [  169.024326] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.882Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1195vm-test-run-nixbot-gitlab> gitlab # [  169.027128] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.885Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1196vm-test-run-nixbot-gitlab> gitlab # [  169.030156] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.887Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1197vm-test-run-nixbot-gitlab> gitlab # [  169.032666] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.890Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1198vm-test-run-nixbot-gitlab> gitlab # [  169.035082] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.892Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1199vm-test-run-nixbot-gitlab> gitlab # [  169.037639] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.894Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1200vm-test-run-nixbot-gitlab> gitlab # [  169.039894] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.897Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1201vm-test-run-nixbot-gitlab> gitlab # [  169.042217] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.900Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1202vm-test-run-nixbot-gitlab> gitlab # [  169.044464] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.903Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1203vm-test-run-nixbot-gitlab> gitlab # [  169.046626] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.906Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1204vm-test-run-nixbot-gitlab> gitlab # [  169.048954] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.908Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1205vm-test-run-nixbot-gitlab> gitlab # [  169.051003] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.911Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1206vm-test-run-nixbot-gitlab> gitlab # [  169.053409] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.914Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1207vm-test-run-nixbot-gitlab> gitlab # [  169.055430] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.916Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1208vm-test-run-nixbot-gitlab> gitlab # [  169.057697] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.919Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1209vm-test-run-nixbot-gitlab> gitlab # [  169.059735] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.920Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1210vm-test-run-nixbot-gitlab> gitlab # [  169.061891] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.922Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1211vm-test-run-nixbot-gitlab> gitlab # [  169.063870] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.925Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1212vm-test-run-nixbot-gitlab> gitlab # [  169.065947] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.927Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1213vm-test-run-nixbot-gitlab> gitlab # [  169.067888] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.929Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1214vm-test-run-nixbot-gitlab> gitlab # [  169.069968] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.931Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1215vm-test-run-nixbot-gitlab> gitlab # [  169.072019] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.933Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1216vm-test-run-nixbot-gitlab> gitlab # [  169.074014] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.935Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1217vm-test-run-nixbot-gitlab> gitlab # [  169.076038] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.937Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1218vm-test-run-nixbot-gitlab> gitlab # [  169.077968] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.939Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1219vm-test-run-nixbot-gitlab> gitlab # [  169.080029] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.941Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1220vm-test-run-nixbot-gitlab> gitlab # [  169.083283] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.944Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1221vm-test-run-nixbot-gitlab> gitlab # [  169.085366] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.945Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1222vm-test-run-nixbot-gitlab> gitlab # [  169.087294] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.948Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1223vm-test-run-nixbot-gitlab> gitlab # [  169.089293] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.950Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1224vm-test-run-nixbot-gitlab> gitlab # [  169.091195] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.952Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1225vm-test-run-nixbot-gitlab> gitlab # [  169.093226] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.955Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1226vm-test-run-nixbot-gitlab> gitlab # [  169.095134] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:41.958Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1227vm-test-run-nixbot-gitlab> gitlab # [  169.123533] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.024Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1228vm-test-run-nixbot-gitlab> gitlab # [  169.128033] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.028Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1229vm-test-run-nixbot-gitlab> gitlab # [  169.131939] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.030Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1230vm-test-run-nixbot-gitlab> gitlab # [  169.136121] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.033Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1231vm-test-run-nixbot-gitlab> gitlab # [  169.139980] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.035Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1232vm-test-run-nixbot-gitlab> gitlab # [  169.144098] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.039Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1233vm-test-run-nixbot-gitlab> gitlab # [  169.147810] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.041Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1234vm-test-run-nixbot-gitlab> gitlab # [  169.151607] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.044Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1235vm-test-run-nixbot-gitlab> gitlab # [  169.154952] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.046Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1236vm-test-run-nixbot-gitlab> gitlab # [  169.158029] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.049Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1237vm-test-run-nixbot-gitlab> gitlab # [  169.161098] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.052Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1238vm-test-run-nixbot-gitlab> gitlab # [  169.164054] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.056Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1239vm-test-run-nixbot-gitlab> gitlab # [  169.167014] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.058Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1240vm-test-run-nixbot-gitlab> gitlab # [  169.169960] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.062Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1241vm-test-run-nixbot-gitlab> gitlab # [  169.172631] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.063Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1242vm-test-run-nixbot-gitlab> gitlab # [  169.175285] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.066Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1243vm-test-run-nixbot-gitlab> gitlab # [  169.178169] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.067Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1244vm-test-run-nixbot-gitlab> gitlab # [  169.180922] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.070Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1245vm-test-run-nixbot-gitlab> gitlab # [  169.183431] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.072Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [  169.186007] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.073Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [  169.188465] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.076Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [  169.190895] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.077Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [  169.193394] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.080Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [  169.195718] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.082Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [  169.198050] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.084Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [  169.200436] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.086Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [  169.202522] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.088Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [  169.204656] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.090Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [  169.206895] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.092Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [  169.209300] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.094Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [  169.211405] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.097Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [  169.213605] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.099Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [  169.215763] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.102Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [  169.217985] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.104Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [  169.220167] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.107Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [  169.222247] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.109Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [  169.224415] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.111Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  169.226490] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.113Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  169.228663] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.116Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  169.230732] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.118Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  169.232917] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.120Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  169.235148] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.122Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  169.237072] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.125Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  169.239195] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.128Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  169.241383] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.130Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  169.243415] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.132Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  169.245613] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.145Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  169.247693] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.147Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  169.249914] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.149Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  169.252210] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.153Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  169.254750] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.155Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  169.257605] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.158Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  169.259892] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.160Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  169.262301] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.162Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  169.264333] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.164Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  169.266660] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.167Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  169.268788] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:27:42.169Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  170.322249] gitlab-workhorse[1286]: time="2026-06-14T14:27:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1285vm-test-run-nixbot-gitlab> gitlab # [  180.330612] gitlab-workhorse[1286]: time="2026-06-14T14:27:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1286vm-test-run-nixbot-gitlab> gitlab # [  190.331773] gitlab-workhorse[1286]: time="2026-06-14T14:28:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1287vm-test-run-nixbot-gitlab> gitlab # [  200.342483] gitlab-workhorse[1286]: time="2026-06-14T14:28:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1288vm-test-run-nixbot-gitlab> gitlab # [  207.324832] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:20.224Z","message":"Booted Rails 7.2.3 application in production environment"}1289vm-test-run-nixbot-gitlab> gitlab # [  207.325302] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:20.225Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1290vm-test-run-nixbot-gitlab> gitlab # [  207.325458] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:20.225Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1291vm-test-run-nixbot-gitlab> gitlab # [  207.325578] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:20.225Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1292vm-test-run-nixbot-gitlab> gitlab # [  207.327656] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:20.228Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1293vm-test-run-nixbot-gitlab> gitlab # [  209.948542] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:22.847Z","message":"Cleaning working queues"}1294vm-test-run-nixbot-gitlab> gitlab # [  209.957929] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:22.857Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4f398505ccc8870ce987537c","created_at":"2026-06-14T14:26:49.060Z","trace_propagation_headers":{"sentry-trace":"a38e4abb5c44419ea95202983a0d3542-9a81a74c7157408e","baggage":"sentry-trace_id=a38e4abb5c44419ea95202983a0d3542,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"c14826d9a694f4fba3ffac61225f3616","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T14:26:49.071Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4f398505ccc8870ce987537c: start","job_status":"start","queue_duration_s":93.786237,"scheduling_latency_s":93.786288}1295vm-test-run-nixbot-gitlab> gitlab # [  210.117301] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.016Z","pid":2369,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1296vm-test-run-nixbot-gitlab> gitlab # [  210.122722] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.017Z","pid":2369,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1297vm-test-run-nixbot-gitlab> gitlab # [  210.127319] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.017Z","pid":2369,"message":"! #\u003cThread:0x0000e2f46679c000 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1298vm-test-run-nixbot-gitlab> gitlab # [  210.139807] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.017Z","pid":2369,"message":"! #\u003cThread:0x0000e2f4669bff80 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1299vm-test-run-nixbot-gitlab> gitlab # [  210.149993] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.017Z","pid":2369,"message":"Use Ctrl-C to stop"}1300vm-test-run-nixbot-gitlab> gitlab # [  210.152297] bundle[2369]: {"timestamp":"2026-06-14T14:28:23.017Z","pid":2369,"message":"Sending status to systemd every 1.0 sec"}1301vm-test-run-nixbot-gitlab> gitlab # [  210.341688] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:23.240Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4f398505ccc8870ce987537c","created_at":"2026-06-14T14:26:49.060Z","trace_propagation_headers":{"sentry-trace":"a38e4abb5c44419ea95202983a0d3542-9a81a74c7157408e","baggage":"sentry-trace_id=a38e4abb5c44419ea95202983a0d3542,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"c14826d9a694f4fba3ffac61225f3616","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T14:26:49.071Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4f398505ccc8870ce987537c: done: 0.383634 sec","job_status":"done","queue_duration_s":93.786237,"scheduling_latency_s":93.786288,"redis_calls":24,"redis_duration_s":0.009869,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004653,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003218,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.21205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20894374000000002,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.383634,"completed_at":"2026-06-14T14:28:23.240Z","load_balancing_strategy":"primary","db_duration_s":0.103856,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1302vm-test-run-nixbot-gitlab> gitlab # [  211.126555] bundle[2369]: {"timestamp":"2026-06-14T14:28:24.026Z","pid":2369,"message":"- Worker 0 (PID: 3215) booted in 0.94s, phase: 0"}1303vm-test-run-nixbot-gitlab> gitlab # [  211.134925] bundle[2369]: {"timestamp":"2026-06-14T14:28:24.035Z","pid":2369,"message":"- Worker 1 (PID: 3219) booted in 0.93s, phase: 0"}1304vm-test-run-nixbot-gitlab> gitlab # [  211.140462] systemd[1]: Started gitlab.service.1305vm-test-run-nixbot-gitlab> gitlab # [  211.143680] systemd[1]: Reached target Common target for all GitLab services..1306vm-test-run-nixbot-gitlab> gitlab # [  211.150380] systemd[1]: Starting setup-gitlab.service...1307vm-test-run-nixbot-gitlab> gitlab # [  211.177299] setup-gitlab-start[3256]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1308vm-test-run-nixbot-gitlab> gitlab # [  214.199608] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:28:27 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 29841309vm-test-run-nixbot-gitlab> gitlab # [  214.205616] setup-gitlab-start[3279]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1310vm-test-run-nixbot-gitlab> gitlab # [  214.213286] setup-gitlab-start[3280]: ++ jq -r .access_token1311vm-test-run-nixbot-gitlab> gitlab # [  215.232600] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:28:28 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 9991312vm-test-run-nixbot-gitlab> gitlab # [  215.238406] setup-gitlab-start[3256]: + BEARER=e502fa590c4fa78e4abaa79e96310ac3232c33a4e656e2e456bcf5f7c5f361971313vm-test-run-nixbot-gitlab> gitlab # [  215.242492] setup-gitlab-start[3256]: + AUTH='Authorization: Bearer e502fa590c4fa78e4abaa79e96310ac3232c33a4e656e2e456bcf5f7c5f36197'1314vm-test-run-nixbot-gitlab> gitlab # [  215.246966] setup-gitlab-start[3256]: + curl -fs -X PUT -H 'Authorization: Bearer e502fa590c4fa78e4abaa79e96310ac3232c33a4e656e2e456bcf5f7c5f36197' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1315vm-test-run-nixbot-gitlab> gitlab # [  216.460437] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:28:29 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 11911316vm-test-run-nixbot-gitlab> gitlab # [  216.463232] setup-gitlab-start[3256]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1317vm-test-run-nixbot-gitlab> gitlab # [  216.479127] runuser[3302]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1318vm-test-run-nixbot-gitlab> gitlab # [  216.947003] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1319vm-test-run-nixbot-gitlab> gitlab # [  216.953212] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1320vm-test-run-nixbot-gitlab> gitlab # [  216.959082] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1321vm-test-run-nixbot-gitlab> gitlab # [  217.704784] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1322vm-test-run-nixbot-gitlab> gitlab # [  217.705501] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1323vm-test-run-nixbot-gitlab> gitlab # [  217.716806] setup-gitlab-start[3303]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1324vm-test-run-nixbot-gitlab> gitlab # [  222.913477] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:35.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"bb181900ae3f96a7ab6029fc","created_at":"2026-06-14T14:28:35.766Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ccc3ef0d93eca7bdd8ee8b1db3f8ed6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.809Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-bb181900ae3f96a7ab6029fc: start","job_status":"start","queue_duration_s":0.003044,"scheduling_latency_s":0.003106}1325vm-test-run-nixbot-gitlab> gitlab # [  223.094616] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:35.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"bb181900ae3f96a7ab6029fc","created_at":"2026-06-14T14:28:35.766Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ccc3ef0d93eca7bdd8ee8b1db3f8ed6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.809Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-bb181900ae3f96a7ab6029fc: done: 0.18218 sec","job_status":"done","queue_duration_s":0.003044,"scheduling_latency_s":0.003106,"redis_calls":10,"redis_duration_s":0.020479,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002384,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00384,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0012306,"duration_s":0.18218,"completed_at":"2026-06-14T14:28:35.994Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326vm-test-run-nixbot-gitlab> gitlab # [  223.177622] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a64dfd14d1190a67a9163dfb","created_at":"2026-06-14T14:28:35.837Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b330e426c580affdee10dc7ff78f408c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.856Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a64dfd14d1190a67a9163dfb: start","job_status":"start","queue_duration_s":0.144523,"scheduling_latency_s":0.144571}1327vm-test-run-nixbot-gitlab> gitlab # [  223.195712] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a64dfd14d1190a67a9163dfb","created_at":"2026-06-14T14:28:35.837Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b330e426c580affdee10dc7ff78f408c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.856Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a64dfd14d1190a67a9163dfb: done: 0.064029 sec","job_status":"done","queue_duration_s":0.144523,"scheduling_latency_s":0.144571,"redis_calls":12,"redis_duration_s":0.003972,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001376,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008722,"duration_s":0.064029,"completed_at":"2026-06-14T14:28:36.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005333,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328vm-test-run-nixbot-gitlab> gitlab # [  223.243942] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ad157afc322d4b183bd97504","created_at":"2026-06-14T14:28:35.864Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.892Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ad157afc322d4b183bd97504: start","job_status":"start","queue_duration_s":0.194055,"scheduling_latency_s":0.194102}1329vm-test-run-nixbot-gitlab> gitlab # [  223.382106] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ad157afc322d4b183bd97504","created_at":"2026-06-14T14:28:35.864Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:35.892Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ad157afc322d4b183bd97504: done: 0.194479 sec","job_status":"done","queue_duration_s":0.194055,"scheduling_latency_s":0.194102,"redis_calls":19,"redis_duration_s":0.076015,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003583,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.001496,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.067767,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003169,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00242362,"duration_s":0.194479,"completed_at":"2026-06-14T14:28:36.281Z","load_balancing_strategy":"primary","db_duration_s":0.012378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330vm-test-run-nixbot-gitlab> gitlab # [  223.472228] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"eed9ed4a54a30fd51e986703","created_at":"2026-06-14T14:28:35.957Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22859376f7638fc631467d5bc1d2f277","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.035Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-eed9ed4a54a30fd51e986703: start","job_status":"start","queue_duration_s":0.257218,"scheduling_latency_s":0.257267}1331vm-test-run-nixbot-gitlab> gitlab # [  223.524660] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"eed9ed4a54a30fd51e986703","created_at":"2026-06-14T14:28:35.957Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22859376f7638fc631467d5bc1d2f277","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.035Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-eed9ed4a54a30fd51e986703: done: 0.131753 sec","job_status":"done","queue_duration_s":0.257218,"scheduling_latency_s":0.257267,"redis_calls":10,"redis_duration_s":0.054235,"redis_read_bytes":9,"redis_write_bytes":1035,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004199,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.042089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007947,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00160738,"duration_s":0.131753,"completed_at":"2026-06-14T14:28:36.423Z","load_balancing_strategy":"primary","db_duration_s":0.006077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332vm-test-run-nixbot-gitlab> gitlab # [  223.600456] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"76b5a7413cc18a9b2d5d1141","created_at":"2026-06-14T14:28:36.088Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f6412ac202a30ebe39161c9fef95132","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.102Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-76b5a7413cc18a9b2d5d1141: start","job_status":"start","queue_duration_s":0.333699,"scheduling_latency_s":0.333746}1333vm-test-run-nixbot-gitlab> gitlab # [  223.671511] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"76b5a7413cc18a9b2d5d1141","created_at":"2026-06-14T14:28:36.088Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f6412ac202a30ebe39161c9fef95132","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.102Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-76b5a7413cc18a9b2d5d1141: done: 0.134659 sec","job_status":"done","queue_duration_s":0.333699,"scheduling_latency_s":0.333746,"redis_calls":12,"redis_duration_s":0.034876000000000004,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004769,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.026138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003969,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00153604,"exclusive_lock_hold_duration_s":0.02779116,"duration_s":0.134659,"completed_at":"2026-06-14T14:28:36.570Z","load_balancing_strategy":"primary","db_duration_s":0.021427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334vm-test-run-nixbot-gitlab> gitlab # [  223.743015] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ac05a33d5ca838dc738cfd9d","created_at":"2026-06-14T14:28:36.143Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e99cf5e485b6b01f5685d56b3b34c257","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.178Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ac05a33d5ca838dc738cfd9d: start","job_status":"start","queue_duration_s":0.395761,"scheduling_latency_s":0.395812}1335vm-test-run-nixbot-gitlab> gitlab # [  223.760737] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.646Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ac05a33d5ca838dc738cfd9d","created_at":"2026-06-14T14:28:36.143Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e99cf5e485b6b01f5685d56b3b34c257","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.178Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ac05a33d5ca838dc738cfd9d: done: 0.072114 sec","job_status":"done","queue_duration_s":0.395761,"scheduling_latency_s":0.395812,"redis_calls":10,"redis_duration_s":0.002042,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033142,"duration_s":0.072114,"completed_at":"2026-06-14T14:28:36.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006859,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336vm-test-run-nixbot-gitlab> gitlab # [  223.802566] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:36.655Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e742b7f6f5642631c55af5e9","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e742b7f6f5642631c55af5e9: start","job_status":"start","queue_duration_s":0.409146,"scheduling_latency_s":0.40919}1337vm-test-run-nixbot-gitlab> gitlab # [  224.260771] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.160Z","message":"No existing merge request to be cleaned up."}1338vm-test-run-nixbot-gitlab> gitlab # [  224.277844] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.178Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e742b7f6f5642631c55af5e9","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e742b7f6f5642631c55af5e9: done: 0.522652 sec","job_status":"done","queue_duration_s":0.409146,"scheduling_latency_s":0.40919,"redis_calls":13,"redis_duration_s":0.008462,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002336,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.308624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065206,"duration_s":0.522652,"completed_at":"2026-06-14T14:28:37.178Z","load_balancing_strategy":"primary","db_duration_s":0.00721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339vm-test-run-nixbot-gitlab> gitlab # [  224.364176] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.179Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"00aa82aa466c8c13c7d87dee","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-00aa82aa466c8c13c7d87dee: start","job_status":"start","queue_duration_s":0.933017,"scheduling_latency_s":0.933039}1340vm-test-run-nixbot-gitlab> gitlab # [  224.379489] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.191Z","message":"No existing merge request to be cleaned up."}1341vm-test-run-nixbot-gitlab> gitlab # [  224.381497] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.200Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"00aa82aa466c8c13c7d87dee","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-00aa82aa466c8c13c7d87dee: done: 0.021303 sec","job_status":"done","queue_duration_s":0.933017,"scheduling_latency_s":0.933039,"redis_calls":10,"redis_duration_s":0.001529,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021303,"completed_at":"2026-06-14T14:28:37.200Z","load_balancing_strategy":"primary","db_duration_s":0.002507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342vm-test-run-nixbot-gitlab> gitlab # [  224.419489] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.202Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"77f4ea8e6a16152e9663ba5a","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-77f4ea8e6a16152e9663ba5a: start","job_status":"start","queue_duration_s":0.955572,"scheduling_latency_s":0.955592}1343vm-test-run-nixbot-gitlab> gitlab # [  224.431264] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.213Z","message":"No existing merge request to be cleaned up."}1344vm-test-run-nixbot-gitlab> gitlab # [  224.433798] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.222Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"77f4ea8e6a16152e9663ba5a","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-77f4ea8e6a16152e9663ba5a: done: 0.020694 sec","job_status":"done","queue_duration_s":0.955572,"scheduling_latency_s":0.955592,"redis_calls":10,"redis_duration_s":0.001499,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020694,"completed_at":"2026-06-14T14:28:37.222Z","load_balancing_strategy":"primary","db_duration_s":0.002373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345vm-test-run-nixbot-gitlab> gitlab # [  224.470044] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.224Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1ac419e20eb95160cb603537","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1ac419e20eb95160cb603537: start","job_status":"start","queue_duration_s":0.977892,"scheduling_latency_s":0.977913}1346vm-test-run-nixbot-gitlab> gitlab # [  224.481490] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.235Z","message":"No existing merge request to be cleaned up."}1347vm-test-run-nixbot-gitlab> gitlab # [  224.483036] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.243Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1ac419e20eb95160cb603537","created_at":"2026-06-14T14:28:36.144Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-acec833978db4603","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e2524a5bd0c8093f05171012ef82cc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.246Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1ac419e20eb95160cb603537: done: 0.019004 sec","job_status":"done","queue_duration_s":0.977892,"scheduling_latency_s":0.977913,"redis_calls":10,"redis_duration_s":0.001348,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019004,"completed_at":"2026-06-14T14:28:37.243Z","load_balancing_strategy":"primary","db_duration_s":0.002208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348vm-test-run-nixbot-gitlab> gitlab # [  224.519138] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2fd2da97177e875fb012fde4","created_at":"2026-06-14T14:28:36.247Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7d706d23ad68955fc5aa40edde6a03f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.277Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2fd2da97177e875fb012fde4: start","job_status":"start","queue_duration_s":0.967274,"scheduling_latency_s":0.967292}1349vm-test-run-nixbot-gitlab> gitlab # [  224.532706] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2fd2da97177e875fb012fde4","created_at":"2026-06-14T14:28:36.247Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7d706d23ad68955fc5aa40edde6a03f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.277Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2fd2da97177e875fb012fde4: done: 0.025492 sec","job_status":"done","queue_duration_s":0.967274,"scheduling_latency_s":0.967292,"redis_calls":12,"redis_duration_s":0.00168,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000647,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048928,"exclusive_lock_hold_duration_s":0.0061372,"duration_s":0.025492,"completed_at":"2026-06-14T14:28:37.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350vm-test-run-nixbot-gitlab> gitlab # [  224.572148] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0fc97b615652b71f5ee5ca95","created_at":"2026-06-14T14:28:36.294Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6fdbb1704d2cd689fac14b5b20071d3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.311Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0fc97b615652b71f5ee5ca95: start","job_status":"start","queue_duration_s":0.960035,"scheduling_latency_s":0.960052}1351vm-test-run-nixbot-gitlab> gitlab # [  224.585442] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0fc97b615652b71f5ee5ca95","created_at":"2026-06-14T14:28:36.294Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6fdbb1704d2cd689fac14b5b20071d3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.311Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0fc97b615652b71f5ee5ca95: done: 0.029316 sec","job_status":"done","queue_duration_s":0.960035,"scheduling_latency_s":0.960052,"redis_calls":12,"redis_duration_s":0.003721,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000565,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055318,"exclusive_lock_hold_duration_s":0.00740634,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029316,"completed_at":"2026-06-14T14:28:37.300Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352vm-test-run-nixbot-gitlab> gitlab # [  224.625691] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.302Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"be2950e3bcf32bad876b973b","created_at":"2026-06-14T14:28:36.370Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6b7d5017bf72ec875841146b0ae005d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.397Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-be2950e3bcf32bad876b973b: start","job_status":"start","queue_duration_s":0.90433,"scheduling_latency_s":0.904346}1353vm-test-run-nixbot-gitlab> gitlab # [  224.638895] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"be2950e3bcf32bad876b973b","created_at":"2026-06-14T14:28:36.370Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6b7d5017bf72ec875841146b0ae005d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.397Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-be2950e3bcf32bad876b973b: done: 0.057347 sec","job_status":"done","queue_duration_s":0.90433,"scheduling_latency_s":0.904346,"redis_calls":16,"redis_duration_s":0.004194,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002363,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033748,"duration_s":0.057347,"completed_at":"2026-06-14T14:28:37.359Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354vm-test-run-nixbot-gitlab> gitlab # [  224.678999] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"530fc006ec488d83d49b9037","created_at":"2026-06-14T14:28:36.418Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f339462dc1d60e502bbc062b9238ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.443Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-530fc006ec488d83d49b9037: start","job_status":"start","queue_duration_s":0.918041,"scheduling_latency_s":0.918059}1355vm-test-run-nixbot-gitlab> gitlab # [  224.780541] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"530fc006ec488d83d49b9037","created_at":"2026-06-14T14:28:36.418Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f339462dc1d60e502bbc062b9238ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.443Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-530fc006ec488d83d49b9037: done: 0.31991 sec","job_status":"done","queue_duration_s":0.918041,"scheduling_latency_s":0.918059,"redis_calls":574,"redis_duration_s":0.076234,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.074335,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.277228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042406,"duration_s":0.31991,"completed_at":"2026-06-14T14:28:37.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356vm-test-run-nixbot-gitlab> gitlab # [  224.819995] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5ae3124aaf3346f5c7304e09","created_at":"2026-06-14T14:28:36.486Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fff1b6541cbb94024bdcc02c3f04d8ed","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.521Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5ae3124aaf3346f5c7304e09: start","job_status":"start","queue_duration_s":1.161199,"scheduling_latency_s":1.161217}1357vm-test-run-nixbot-gitlab> gitlab # [  224.861765] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5ae3124aaf3346f5c7304e09","created_at":"2026-06-14T14:28:36.486Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fff1b6541cbb94024bdcc02c3f04d8ed","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.521Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5ae3124aaf3346f5c7304e09: done: 0.079929 sec","job_status":"done","queue_duration_s":1.161199,"scheduling_latency_s":1.161217,"redis_calls":12,"redis_duration_s":0.001968,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000811,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056954,"exclusive_lock_hold_duration_s":0.0491023,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.079929,"completed_at":"2026-06-14T14:28:37.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358vm-test-run-nixbot-gitlab> gitlab # [  224.909340] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"49f9d58cf76e875fe3c9d90d","created_at":"2026-06-14T14:28:36.540Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a48f7da84211328bd011a39dc644611e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.564Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-49f9d58cf76e875fe3c9d90d: start","job_status":"start","queue_duration_s":1.201141,"scheduling_latency_s":1.201163}1359vm-test-run-nixbot-gitlab> gitlab # [  224.922562] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"49f9d58cf76e875fe3c9d90d","created_at":"2026-06-14T14:28:36.540Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a48f7da84211328bd011a39dc644611e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.564Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-49f9d58cf76e875fe3c9d90d: done: 0.026011 sec","job_status":"done","queue_duration_s":1.201141,"scheduling_latency_s":1.201163,"redis_calls":10,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024776,"duration_s":0.026011,"completed_at":"2026-06-14T14:28:37.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003952,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360vm-test-run-nixbot-gitlab> gitlab # [  224.962438] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e4629923acae3fe5b90968d7","created_at":"2026-06-14T14:28:36.594Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea2afe5a33da0e8bb2a01fdf16cb4157","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.619Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e4629923acae3fe5b90968d7: start","job_status":"start","queue_duration_s":1.173556,"scheduling_latency_s":1.173576}1361vm-test-run-nixbot-gitlab> gitlab # [  224.975933] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e4629923acae3fe5b90968d7","created_at":"2026-06-14T14:28:36.594Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea2afe5a33da0e8bb2a01fdf16cb4157","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.619Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e4629923acae3fe5b90968d7: done: 0.04346 sec","job_status":"done","queue_duration_s":1.173556,"scheduling_latency_s":1.173576,"redis_calls":14,"redis_duration_s":0.001938,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004555,"exclusive_lock_hold_duration_s":0.02158938,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.04346,"completed_at":"2026-06-14T14:28:37.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362vm-test-run-nixbot-gitlab> gitlab # [  225.017990] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:37.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6137a1859e7eebb67edcf05c","created_at":"2026-06-14T14:28:36.623Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb1083b0cefaa16b25a8814c24351509","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.653Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6137a1859e7eebb67edcf05c: start","job_status":"start","queue_duration_s":1.1847,"scheduling_latency_s":1.184719}1363vm-test-run-nixbot-gitlab> gitlab # [  225.179187] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6137a1859e7eebb67edcf05c","created_at":"2026-06-14T14:28:36.623Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb1083b0cefaa16b25a8814c24351509","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.653Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6137a1859e7eebb67edcf05c: done: 0.241378 sec","job_status":"done","queue_duration_s":1.1847,"scheduling_latency_s":1.184719,"redis_calls":14,"redis_duration_s":0.002588,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052614,"exclusive_lock_hold_duration_s":0.21914484,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.241378,"completed_at":"2026-06-14T14:28:38.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364vm-test-run-nixbot-gitlab> gitlab # [  225.268457] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b41d1497f17a694f07e9aae8","created_at":"2026-06-14T14:28:36.691Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da2e24d73c1a23da30114f302f40249d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T14:28:36.715Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b41d1497f17a694f07e9aae8: start","job_status":"start","queue_duration_s":1.366525,"scheduling_latency_s":1.366544}1365vm-test-run-nixbot-gitlab> gitlab # [  225.286998] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.093Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1366vm-test-run-nixbot-gitlab> gitlab # [  225.291789] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b41d1497f17a694f07e9aae8","created_at":"2026-06-14T14:28:36.691Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da2e24d73c1a23da30114f302f40249d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T14:28:36.715Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b41d1497f17a694f07e9aae8: done: 0.019631 sec","job_status":"done","queue_duration_s":1.366525,"scheduling_latency_s":1.366544,"redis_calls":10,"redis_duration_s":0.002785,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000788,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031424,"duration_s":0.019631,"completed_at":"2026-06-14T14:28:38.101Z","load_balancing_strategy":"primary","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367vm-test-run-nixbot-gitlab> gitlab # [  225.337458] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.102Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4b94d3e6b05b5d411f8cdee3","created_at":"2026-06-14T14:28:36.737Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfe4cf364b92d6761ab96ee1ca9adc1d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.748Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4b94d3e6b05b5d411f8cdee3: start","job_status":"start","queue_duration_s":1.354643,"scheduling_latency_s":1.354659}1368vm-test-run-nixbot-gitlab> gitlab # [  225.349781] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4b94d3e6b05b5d411f8cdee3","created_at":"2026-06-14T14:28:36.737Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfe4cf364b92d6761ab96ee1ca9adc1d","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.748Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4b94d3e6b05b5d411f8cdee3: done: 0.019484 sec","job_status":"done","queue_duration_s":1.354643,"scheduling_latency_s":1.354659,"redis_calls":12,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000574,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003137,"duration_s":0.019484,"completed_at":"2026-06-14T14:28:38.122Z","load_balancing_strategy":"primary","db_duration_s":0.000525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369vm-test-run-nixbot-gitlab> gitlab # [  225.389418] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2d5a41cc01efaae293395cba","created_at":"2026-06-14T14:28:36.755Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0658fa019791191ab099166779b2814","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.765Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2d5a41cc01efaae293395cba: start","job_status":"start","queue_duration_s":1.357909,"scheduling_latency_s":1.357925}1370vm-test-run-nixbot-gitlab> gitlab # [  225.403174] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2d5a41cc01efaae293395cba","created_at":"2026-06-14T14:28:36.755Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0658fa019791191ab099166779b2814","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.765Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2d5a41cc01efaae293395cba: done: 0.019739 sec","job_status":"done","queue_duration_s":1.357909,"scheduling_latency_s":1.357925,"redis_calls":10,"redis_duration_s":0.002316,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000473,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034174,"duration_s":0.019739,"completed_at":"2026-06-14T14:28:38.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371vm-test-run-nixbot-gitlab> gitlab # [  225.442871] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"96bf7a5cbea18aab32802e29","created_at":"2026-06-14T14:28:36.772Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.780Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-96bf7a5cbea18aab32802e29: start","job_status":"start","queue_duration_s":1.364634,"scheduling_latency_s":1.364675}1372vm-test-run-nixbot-gitlab> gitlab # [  225.455693] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"96bf7a5cbea18aab32802e29","created_at":"2026-06-14T14:28:36.772Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.780Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-96bf7a5cbea18aab32802e29: done: 0.134555 sec","job_status":"done","queue_duration_s":1.364634,"scheduling_latency_s":1.364675,"redis_calls":71,"redis_duration_s":0.017804,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002451,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001098,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013536,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031728,"duration_s":0.134555,"completed_at":"2026-06-14T14:28:38.279Z","load_balancing_strategy":"primary","db_duration_s":0.02395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373vm-test-run-nixbot-gitlab> gitlab # [  225.496799] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d87936d5695f6121b6a48965","created_at":"2026-06-14T14:28:36.787Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e45929fb017276cf92f648034f426ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.794Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d87936d5695f6121b6a48965: start","job_status":"start","queue_duration_s":1.48686,"scheduling_latency_s":1.486878}1374vm-test-run-nixbot-gitlab> gitlab # [  225.510180] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.294Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1375vm-test-run-nixbot-gitlab> gitlab # [  225.513174] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.301Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d87936d5695f6121b6a48965","created_at":"2026-06-14T14:28:36.787Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e45929fb017276cf92f648034f426ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.794Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d87936d5695f6121b6a48965: done: 0.020237 sec","job_status":"done","queue_duration_s":1.48686,"scheduling_latency_s":1.486878,"redis_calls":10,"redis_duration_s":0.0024850000000000002,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003245,"duration_s":0.020237,"completed_at":"2026-06-14T14:28:38.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376vm-test-run-nixbot-gitlab> gitlab # [  225.552941] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9f59ddd2614927360036a0c2","created_at":"2026-06-14T14:28:36.802Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbcc2e863c0c586b8623fe726484e03f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.810Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9f59ddd2614927360036a0c2: start","job_status":"start","queue_duration_s":1.492788,"scheduling_latency_s":1.492805}1377vm-test-run-nixbot-gitlab> gitlab # [  225.565733] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9f59ddd2614927360036a0c2","created_at":"2026-06-14T14:28:36.802Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbcc2e863c0c586b8623fe726484e03f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.810Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9f59ddd2614927360036a0c2: done: 0.045063 sec","job_status":"done","queue_duration_s":1.492788,"scheduling_latency_s":1.492805,"redis_calls":10,"redis_duration_s":0.002589,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043088,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.045063,"completed_at":"2026-06-14T14:28:38.347Z","load_balancing_strategy":"primary","db_duration_s":0.023104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378vm-test-run-nixbot-gitlab> gitlab # [  225.605809] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"68b2324f8751e68f9ac6a61c","created_at":"2026-06-14T14:28:36.835Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83bc9c59906b78dbde415ce0cd350988","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.844Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-68b2324f8751e68f9ac6a61c: start","job_status":"start","queue_duration_s":1.505457,"scheduling_latency_s":1.505474}1379vm-test-run-nixbot-gitlab> gitlab # [  225.619461] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"68b2324f8751e68f9ac6a61c","created_at":"2026-06-14T14:28:36.835Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83bc9c59906b78dbde415ce0cd350988","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.844Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-68b2324f8751e68f9ac6a61c: done: 0.035859 sec","job_status":"done","queue_duration_s":1.505457,"scheduling_latency_s":1.505474,"redis_calls":13,"redis_duration_s":0.003564,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001834,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01297758,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.035859,"completed_at":"2026-06-14T14:28:38.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006439,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1380vm-test-run-nixbot-gitlab> gitlab # [  225.661735] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b7357bfec3c75eca01d11dbe","created_at":"2026-06-14T14:28:36.856Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c63fb0ace6f9e9766d90aa283b98123","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T14:28:36.863Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b7357bfec3c75eca01d11dbe: start","job_status":"start","queue_duration_s":1.523599,"scheduling_latency_s":1.523616}1381vm-test-run-nixbot-gitlab> gitlab # [  225.673999] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b7357bfec3c75eca01d11dbe","created_at":"2026-06-14T14:28:36.856Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c63fb0ace6f9e9766d90aa283b98123","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T14:28:36.863Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b7357bfec3c75eca01d11dbe: done: 0.033416 sec","job_status":"done","queue_duration_s":1.523599,"scheduling_latency_s":1.523616,"redis_calls":10,"redis_duration_s":0.002639,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036792,"duration_s":0.033416,"completed_at":"2026-06-14T14:28:38.420Z","load_balancing_strategy":"primary","db_duration_s":0.005896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382vm-test-run-nixbot-gitlab> gitlab # [  225.712538] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a5f45eca18f426d976981028","created_at":"2026-06-14T14:28:36.870Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c33b4f64e6913cf56a345ec662de6062","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T14:28:36.876Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a5f45eca18f426d976981028: start","job_status":"start","queue_duration_s":1.544918,"scheduling_latency_s":1.544934}1383vm-test-run-nixbot-gitlab> gitlab # [  225.724977] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.437Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1384vm-test-run-nixbot-gitlab> gitlab # [  225.728358] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a5f45eca18f426d976981028","created_at":"2026-06-14T14:28:36.870Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c33b4f64e6913cf56a345ec662de6062","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T14:28:36.876Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a5f45eca18f426d976981028: done: 0.022954 sec","job_status":"done","queue_duration_s":1.544918,"scheduling_latency_s":1.544934,"redis_calls":12,"redis_duration_s":0.005322,"redis_read_bytes":171,"redis_write_bytes":1346,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003444,"redis_feature_flag_read_bytes":164,"redis_feature_flag_write_bytes":444,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001416,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031536,"duration_s":0.022954,"completed_at":"2026-06-14T14:28:38.444Z","load_balancing_strategy":"primary","db_duration_s":0.000651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385vm-test-run-nixbot-gitlab> gitlab # [  225.766759] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e749bc97258e77ddf41671a6","created_at":"2026-06-14T14:28:36.883Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"debcf977f6220e25b4dfeee76f735104","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.889Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e749bc97258e77ddf41671a6: start","job_status":"start","queue_duration_s":1.556651,"scheduling_latency_s":1.556668}1386vm-test-run-nixbot-gitlab> gitlab # [  225.780376] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.457Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1387vm-test-run-nixbot-gitlab> gitlab # [  225.783421] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e749bc97258e77ddf41671a6","created_at":"2026-06-14T14:28:36.883Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"debcf977f6220e25b4dfeee76f735104","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.889Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e749bc97258e77ddf41671a6: done: 0.018427 sec","job_status":"done","queue_duration_s":1.556651,"scheduling_latency_s":1.556668,"redis_calls":10,"redis_duration_s":0.0023639999999999998,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000544,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031232,"duration_s":0.018427,"completed_at":"2026-06-14T14:28:38.464Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388vm-test-run-nixbot-gitlab> gitlab # [  225.823700] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9cbd6c3d7b7ef54e03186e2c","created_at":"2026-06-14T14:28:36.894Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"526abc66c7196a1e17880aa8b3542d24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.900Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9cbd6c3d7b7ef54e03186e2c: start","job_status":"start","queue_duration_s":1.565596,"scheduling_latency_s":1.565614}1389vm-test-run-nixbot-gitlab> gitlab # [  225.837473] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.477Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1390vm-test-run-nixbot-gitlab> gitlab # [  225.840942] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9cbd6c3d7b7ef54e03186e2c","created_at":"2026-06-14T14:28:36.894Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"526abc66c7196a1e17880aa8b3542d24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.900Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9cbd6c3d7b7ef54e03186e2c: done: 0.018461 sec","job_status":"done","queue_duration_s":1.565596,"scheduling_latency_s":1.565614,"redis_calls":10,"redis_duration_s":0.0023439999999999997,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00054,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031156,"duration_s":0.018461,"completed_at":"2026-06-14T14:28:38.484Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391vm-test-run-nixbot-gitlab> gitlab # [  225.882006] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"67e5d95573c8e64b5efbf7bf","created_at":"2026-06-14T14:28:36.906Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89402cd5cc084dfe20a1718391997d37","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.912Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-67e5d95573c8e64b5efbf7bf: start","job_status":"start","queue_duration_s":1.573894,"scheduling_latency_s":1.573911}1392vm-test-run-nixbot-gitlab> gitlab # [  225.895298] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"67e5d95573c8e64b5efbf7bf","created_at":"2026-06-14T14:28:36.906Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89402cd5cc084dfe20a1718391997d37","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.912Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-67e5d95573c8e64b5efbf7bf: done: 0.218704 sec","job_status":"done","queue_duration_s":1.573894,"scheduling_latency_s":1.573911,"redis_calls":12,"redis_duration_s":0.002568,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.199664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005528,"exclusive_lock_hold_duration_s":0.19829712,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.218704,"completed_at":"2026-06-14T14:28:38.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01446,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1393vm-test-run-nixbot-gitlab> gitlab # [  225.936248] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c343773d75e66523110214b7","created_at":"2026-06-14T14:28:36.917Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3de0609dff56d004b0fa11d512aa4a07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.924Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c343773d75e66523110214b7: start","job_status":"start","queue_duration_s":1.782374,"scheduling_latency_s":1.782391}1394vm-test-run-nixbot-gitlab> gitlab # [  225.949666] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.719Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1395vm-test-run-nixbot-gitlab> gitlab # [  225.953009] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c343773d75e66523110214b7","created_at":"2026-06-14T14:28:36.917Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3de0609dff56d004b0fa11d512aa4a07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T14:28:36.924Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c343773d75e66523110214b7: done: 0.020288 sec","job_status":"done","queue_duration_s":1.782374,"scheduling_latency_s":1.782391,"redis_calls":10,"redis_duration_s":0.001633,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023054,"duration_s":0.020288,"completed_at":"2026-06-14T14:28:38.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396vm-test-run-nixbot-gitlab> gitlab # [  225.992210] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"739c18005843c6f4d51214fa","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-739c18005843c6f4d51214fa: start","job_status":"start","queue_duration_s":0.457115,"scheduling_latency_s":0.457133}1397vm-test-run-nixbot-gitlab> gitlab # [  226.003658] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"739c18005843c6f4d51214fa","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-739c18005843c6f4d51214fa: done: 0.021194 sec","job_status":"done","queue_duration_s":0.457115,"scheduling_latency_s":0.457133,"redis_calls":16,"redis_duration_s":0.002133,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021988,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021194,"completed_at":"2026-06-14T14:28:38.749Z","load_balancing_strategy":"primary","db_duration_s":0.000359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398vm-test-run-nixbot-gitlab> gitlab # [  226.042138] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9133cc0f5ba5b8edd7b0f9c","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9133cc0f5ba5b8edd7b0f9c: start","job_status":"start","queue_duration_s":0.479523,"scheduling_latency_s":0.47954}1399vm-test-run-nixbot-gitlab> gitlab # [  226.053607] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9133cc0f5ba5b8edd7b0f9c","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9133cc0f5ba5b8edd7b0f9c: done: 0.015308 sec","job_status":"done","queue_duration_s":0.479523,"scheduling_latency_s":0.47954,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015308,"completed_at":"2026-06-14T14:28:38.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400vm-test-run-nixbot-gitlab> gitlab # [  226.091292] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a1ee7cdf52c95368aa015c1","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a1ee7cdf52c95368aa015c1: start","job_status":"start","queue_duration_s":0.496096,"scheduling_latency_s":0.496114}1401vm-test-run-nixbot-gitlab> gitlab # [  226.103268] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a1ee7cdf52c95368aa015c1","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a1ee7cdf52c95368aa015c1: done: 0.014996 sec","job_status":"done","queue_duration_s":0.496096,"scheduling_latency_s":0.496114,"redis_calls":13,"redis_duration_s":0.001648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014996,"completed_at":"2026-06-14T14:28:38.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402vm-test-run-nixbot-gitlab> gitlab # [  226.140222] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f152075acbce615af5f6e095","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f152075acbce615af5f6e095: start","job_status":"start","queue_duration_s":0.512337,"scheduling_latency_s":0.512354}1403vm-test-run-nixbot-gitlab> gitlab # [  226.151820] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f152075acbce615af5f6e095","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f152075acbce615af5f6e095: done: 0.018025 sec","job_status":"done","queue_duration_s":0.512337,"scheduling_latency_s":0.512354,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018025,"completed_at":"2026-06-14T14:28:38.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404vm-test-run-nixbot-gitlab> gitlab # [  226.188785] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59be766ae59ae6ce93607f8","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59be766ae59ae6ce93607f8: start","job_status":"start","queue_duration_s":0.531613,"scheduling_latency_s":0.531631}1405vm-test-run-nixbot-gitlab> gitlab # [  226.200475] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59be766ae59ae6ce93607f8","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59be766ae59ae6ce93607f8: done: 0.014629 sec","job_status":"done","queue_duration_s":0.531613,"scheduling_latency_s":0.531631,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014629,"completed_at":"2026-06-14T14:28:38.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406vm-test-run-nixbot-gitlab> gitlab # [  226.237013] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b1a5f7462b689454f75f7d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b1a5f7462b689454f75f7d: start","job_status":"start","queue_duration_s":0.547517,"scheduling_latency_s":0.547535}1407vm-test-run-nixbot-gitlab> gitlab # [  226.248665] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b1a5f7462b689454f75f7d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b1a5f7462b689454f75f7d: done: 0.015219 sec","job_status":"done","queue_duration_s":0.547517,"scheduling_latency_s":0.547535,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015219,"completed_at":"2026-06-14T14:28:38.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408vm-test-run-nixbot-gitlab> gitlab # [  226.286604] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cd076004eee595e63607f63","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cd076004eee595e63607f63: start","job_status":"start","queue_duration_s":0.564129,"scheduling_latency_s":0.564148}1409vm-test-run-nixbot-gitlab> gitlab # [  226.298498] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cd076004eee595e63607f63","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cd076004eee595e63607f63: done: 0.014945 sec","job_status":"done","queue_duration_s":0.564129,"scheduling_latency_s":0.564148,"redis_calls":13,"redis_duration_s":0.0014399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014945,"completed_at":"2026-06-14T14:28:38.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [  226.335400] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137ac9c641cfb4bc7ef68f29","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137ac9c641cfb4bc7ef68f29: start","job_status":"start","queue_duration_s":0.580031,"scheduling_latency_s":0.580049}1411vm-test-run-nixbot-gitlab> gitlab # [  226.347017] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137ac9c641cfb4bc7ef68f29","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137ac9c641cfb4bc7ef68f29: done: 0.014759 sec","job_status":"done","queue_duration_s":0.580031,"scheduling_latency_s":0.580049,"redis_calls":13,"redis_duration_s":0.001658,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014759,"completed_at":"2026-06-14T14:28:38.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [  226.383623] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27c86d716a6cbb0483134e9","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27c86d716a6cbb0483134e9: start","job_status":"start","queue_duration_s":0.595854,"scheduling_latency_s":0.595872}1413vm-test-run-nixbot-gitlab> gitlab # [  226.395511] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27c86d716a6cbb0483134e9","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27c86d716a6cbb0483134e9: done: 0.01478 sec","job_status":"done","queue_duration_s":0.595854,"scheduling_latency_s":0.595872,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01478,"completed_at":"2026-06-14T14:28:38.881Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [  226.432049] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3c16ab4462119002011ccea","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3c16ab4462119002011ccea: start","job_status":"start","queue_duration_s":0.611871,"scheduling_latency_s":0.61189}1415vm-test-run-nixbot-gitlab> gitlab # [  226.443412] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3c16ab4462119002011ccea","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3c16ab4462119002011ccea: done: 0.015058 sec","job_status":"done","queue_duration_s":0.611871,"scheduling_latency_s":0.61189,"redis_calls":13,"redis_duration_s":0.001627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015058,"completed_at":"2026-06-14T14:28:38.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  226.481051] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bfc9dd20ee79bc68960611e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bfc9dd20ee79bc68960611e: start","job_status":"start","queue_duration_s":0.628002,"scheduling_latency_s":0.62802}1417vm-test-run-nixbot-gitlab> gitlab # [  226.492708] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bfc9dd20ee79bc68960611e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bfc9dd20ee79bc68960611e: done: 0.015284 sec","job_status":"done","queue_duration_s":0.628002,"scheduling_latency_s":0.62802,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015284,"completed_at":"2026-06-14T14:28:38.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [  226.529337] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59e141d2c81dab1ff6af977f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59e141d2c81dab1ff6af977f: start","job_status":"start","queue_duration_s":0.644366,"scheduling_latency_s":0.644384}1419vm-test-run-nixbot-gitlab> gitlab # [  226.541006] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59e141d2c81dab1ff6af977f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59e141d2c81dab1ff6af977f: done: 0.014744 sec","job_status":"done","queue_duration_s":0.644366,"scheduling_latency_s":0.644384,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014744,"completed_at":"2026-06-14T14:28:38.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420vm-test-run-nixbot-gitlab> gitlab # [  226.577797] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"109b5950dda84e113f425f85","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-109b5950dda84e113f425f85: start","job_status":"start","queue_duration_s":0.660336,"scheduling_latency_s":0.660355}1421vm-test-run-nixbot-gitlab> gitlab # [  226.589514] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"109b5950dda84e113f425f85","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-109b5950dda84e113f425f85: done: 0.014831 sec","job_status":"done","queue_duration_s":0.660336,"scheduling_latency_s":0.660355,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014831,"completed_at":"2026-06-14T14:28:38.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422vm-test-run-nixbot-gitlab> gitlab # [  226.626400] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7555fcf7c66565172d3785","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7555fcf7c66565172d3785: start","job_status":"start","queue_duration_s":0.676385,"scheduling_latency_s":0.676403}1423vm-test-run-nixbot-gitlab> gitlab # [  226.638135] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7555fcf7c66565172d3785","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7555fcf7c66565172d3785: done: 0.014791 sec","job_status":"done","queue_duration_s":0.676385,"scheduling_latency_s":0.676403,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014791,"completed_at":"2026-06-14T14:28:38.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424vm-test-run-nixbot-gitlab> gitlab # [  226.674747] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb5389bd37967e88f21f0321","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb5389bd37967e88f21f0321: start","job_status":"start","queue_duration_s":0.692498,"scheduling_latency_s":0.692517}1425vm-test-run-nixbot-gitlab> gitlab # [  226.687351] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb5389bd37967e88f21f0321","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb5389bd37967e88f21f0321: done: 0.016317 sec","job_status":"done","queue_duration_s":0.692498,"scheduling_latency_s":0.692517,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016317,"completed_at":"2026-06-14T14:28:38.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [  226.724124] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d194e762f41fbbe6d4e00af","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d194e762f41fbbe6d4e00af: start","job_status":"start","queue_duration_s":0.710006,"scheduling_latency_s":0.710025}1427vm-test-run-nixbot-gitlab> gitlab # [  226.735617] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d194e762f41fbbe6d4e00af","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d194e762f41fbbe6d4e00af: done: 0.015196 sec","job_status":"done","queue_duration_s":0.710006,"scheduling_latency_s":0.710025,"redis_calls":13,"redis_duration_s":0.0017230000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015196,"completed_at":"2026-06-14T14:28:38.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428vm-test-run-nixbot-gitlab> gitlab # [  226.772584] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:38.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a781df5a86a4bd668df2d02","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a781df5a86a4bd668df2d02: start","job_status":"start","queue_duration_s":0.726592,"scheduling_latency_s":0.726611}1429vm-test-run-nixbot-gitlab> gitlab # [  226.784200] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a781df5a86a4bd668df2d02","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a781df5a86a4bd668df2d02: done: 0.014832 sec","job_status":"done","queue_duration_s":0.726592,"scheduling_latency_s":0.726611,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014832,"completed_at":"2026-06-14T14:28:39.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430vm-test-run-nixbot-gitlab> gitlab # [  226.821551] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76f2df7b1b96ef5cb0f42e4e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76f2df7b1b96ef5cb0f42e4e: start","job_status":"start","queue_duration_s":0.742704,"scheduling_latency_s":0.742723}1431vm-test-run-nixbot-gitlab> gitlab # [  226.833383] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76f2df7b1b96ef5cb0f42e4e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76f2df7b1b96ef5cb0f42e4e: done: 0.014615 sec","job_status":"done","queue_duration_s":0.742704,"scheduling_latency_s":0.742723,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014615,"completed_at":"2026-06-14T14:28:39.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [  226.870958] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b59e0324bcc7dc410fe762","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b59e0324bcc7dc410fe762: start","job_status":"start","queue_duration_s":0.7584,"scheduling_latency_s":0.758418}1433vm-test-run-nixbot-gitlab> gitlab # [  226.882969] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b59e0324bcc7dc410fe762","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b59e0324bcc7dc410fe762: done: 0.014745 sec","job_status":"done","queue_duration_s":0.7584,"scheduling_latency_s":0.758418,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014745,"completed_at":"2026-06-14T14:28:39.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434vm-test-run-nixbot-gitlab> gitlab # [  226.921044] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee4a9fb26a59b77c4f2e136","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee4a9fb26a59b77c4f2e136: start","job_status":"start","queue_duration_s":0.774353,"scheduling_latency_s":0.774369}1435vm-test-run-nixbot-gitlab> gitlab # [  226.933691] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee4a9fb26a59b77c4f2e136","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee4a9fb26a59b77c4f2e136: done: 0.015029 sec","job_status":"done","queue_duration_s":0.774353,"scheduling_latency_s":0.774369,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015029,"completed_at":"2026-06-14T14:28:39.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436vm-test-run-nixbot-gitlab> gitlab # [  226.970555] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecb8258102a23ca617820b3a","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecb8258102a23ca617820b3a: start","job_status":"start","queue_duration_s":0.790471,"scheduling_latency_s":0.790488}1437vm-test-run-nixbot-gitlab> gitlab # [  226.982417] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecb8258102a23ca617820b3a","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecb8258102a23ca617820b3a: done: 0.014741 sec","job_status":"done","queue_duration_s":0.790471,"scheduling_latency_s":0.790488,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014741,"completed_at":"2026-06-14T14:28:39.075Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438vm-test-run-nixbot-gitlab> gitlab # [  227.019848] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d4131849eabf06303640665","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d4131849eabf06303640665: start","job_status":"start","queue_duration_s":0.806291,"scheduling_latency_s":0.806308}1439vm-test-run-nixbot-gitlab> gitlab # [  227.031756] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d4131849eabf06303640665","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d4131849eabf06303640665: done: 0.015121 sec","job_status":"done","queue_duration_s":0.806291,"scheduling_latency_s":0.806308,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015121,"completed_at":"2026-06-14T14:28:39.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [  227.068908] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fa94d6c5746638631f09a28","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fa94d6c5746638631f09a28: start","job_status":"start","queue_duration_s":0.822838,"scheduling_latency_s":0.822856}1441vm-test-run-nixbot-gitlab> gitlab # [  227.080657] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fa94d6c5746638631f09a28","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fa94d6c5746638631f09a28: done: 0.014722 sec","job_status":"done","queue_duration_s":0.822838,"scheduling_latency_s":0.822856,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014722,"completed_at":"2026-06-14T14:28:39.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442vm-test-run-nixbot-gitlab> gitlab # [  227.118378] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c70d2137399f54d001197fe1","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c70d2137399f54d001197fe1: start","job_status":"start","queue_duration_s":0.838603,"scheduling_latency_s":0.83862}1443vm-test-run-nixbot-gitlab> gitlab # [  227.130207] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c70d2137399f54d001197fe1","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c70d2137399f54d001197fe1: done: 0.015122 sec","job_status":"done","queue_duration_s":0.838603,"scheduling_latency_s":0.83862,"redis_calls":13,"redis_duration_s":0.0015279999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015122,"completed_at":"2026-06-14T14:28:39.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444vm-test-run-nixbot-gitlab> gitlab # [  227.167256] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c0a4ad09d02e2952527bd2b","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c0a4ad09d02e2952527bd2b: start","job_status":"start","queue_duration_s":0.855066,"scheduling_latency_s":0.855084}1445vm-test-run-nixbot-gitlab> gitlab # [  227.179212] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c0a4ad09d02e2952527bd2b","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c0a4ad09d02e2952527bd2b: done: 0.014663 sec","job_status":"done","queue_duration_s":0.855066,"scheduling_latency_s":0.855084,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014663,"completed_at":"2026-06-14T14:28:39.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [  227.216251] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50584654b74df0cce60e3b99","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50584654b74df0cce60e3b99: start","job_status":"start","queue_duration_s":0.871001,"scheduling_latency_s":0.871019}1447vm-test-run-nixbot-gitlab> gitlab # [  227.227800] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50584654b74df0cce60e3b99","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50584654b74df0cce60e3b99: done: 0.014684 sec","job_status":"done","queue_duration_s":0.871001,"scheduling_latency_s":0.871019,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014684,"completed_at":"2026-06-14T14:28:39.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448vm-test-run-nixbot-gitlab> gitlab # [  227.264700] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90ced5f2eaf3a5b73be9c5ac","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90ced5f2eaf3a5b73be9c5ac: start","job_status":"start","queue_duration_s":0.886931,"scheduling_latency_s":0.886948}1449vm-test-run-nixbot-gitlab> gitlab # [  227.276300] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90ced5f2eaf3a5b73be9c5ac","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90ced5f2eaf3a5b73be9c5ac: done: 0.015076 sec","job_status":"done","queue_duration_s":0.886931,"scheduling_latency_s":0.886948,"redis_calls":13,"redis_duration_s":0.001767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015076,"completed_at":"2026-06-14T14:28:39.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [  227.314172] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"651d31584262c6e6fadd7ab0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-651d31584262c6e6fadd7ab0: start","job_status":"start","queue_duration_s":0.903305,"scheduling_latency_s":0.903323}1451vm-test-run-nixbot-gitlab> gitlab # [  227.325973] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"651d31584262c6e6fadd7ab0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-651d31584262c6e6fadd7ab0: done: 0.014514 sec","job_status":"done","queue_duration_s":0.903305,"scheduling_latency_s":0.903323,"redis_calls":13,"redis_duration_s":0.001606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014514,"completed_at":"2026-06-14T14:28:39.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [  227.362930] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b161faa88e3e124e6e0ac73f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b161faa88e3e124e6e0ac73f: start","job_status":"start","queue_duration_s":0.9189,"scheduling_latency_s":0.918917}1453vm-test-run-nixbot-gitlab> gitlab # [  227.374709] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b161faa88e3e124e6e0ac73f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b161faa88e3e124e6e0ac73f: done: 0.151833 sec","job_status":"done","queue_duration_s":0.9189,"scheduling_latency_s":0.918917,"redis_calls":13,"redis_duration_s":0.001975,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.151833,"completed_at":"2026-06-14T14:28:39.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [  227.411585] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"803b6faa8850f1a7e7d05a0e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-803b6faa8850f1a7e7d05a0e: start","job_status":"start","queue_duration_s":1.074444,"scheduling_latency_s":1.074488}1455vm-test-run-nixbot-gitlab> gitlab # [  227.423538] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"803b6faa8850f1a7e7d05a0e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-803b6faa8850f1a7e7d05a0e: done: 0.038931 sec","job_status":"done","queue_duration_s":1.074444,"scheduling_latency_s":1.074488,"redis_calls":13,"redis_duration_s":0.004555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038931,"completed_at":"2026-06-14T14:28:39.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456vm-test-run-nixbot-gitlab> gitlab # [  227.460597] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669f4d88f4c4205e4aea81be","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669f4d88f4c4205e4aea81be: start","job_status":"start","queue_duration_s":1.116069,"scheduling_latency_s":1.116106}1457vm-test-run-nixbot-gitlab> gitlab # [  227.472260] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669f4d88f4c4205e4aea81be","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669f4d88f4c4205e4aea81be: done: 0.028129 sec","job_status":"done","queue_duration_s":1.116069,"scheduling_latency_s":1.116106,"redis_calls":13,"redis_duration_s":0.003525,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002333,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028129,"completed_at":"2026-06-14T14:28:39.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458vm-test-run-nixbot-gitlab> gitlab # [  227.509779] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a2e0618a5a9529c7b8fb86a","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a2e0618a5a9529c7b8fb86a: start","job_status":"start","queue_duration_s":1.146012,"scheduling_latency_s":1.146035}1459vm-test-run-nixbot-gitlab> gitlab # [  227.521483] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a2e0618a5a9529c7b8fb86a","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a2e0618a5a9529c7b8fb86a: done: 0.021263 sec","job_status":"done","queue_duration_s":1.146012,"scheduling_latency_s":1.146035,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021263,"completed_at":"2026-06-14T14:28:39.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460vm-test-run-nixbot-gitlab> gitlab # [  227.558332] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658129f717f7c797a29fc859","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658129f717f7c797a29fc859: start","job_status":"start","queue_duration_s":1.16902,"scheduling_latency_s":1.169039}1461vm-test-run-nixbot-gitlab> gitlab # [  227.570018] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658129f717f7c797a29fc859","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658129f717f7c797a29fc859: done: 0.017494 sec","job_status":"done","queue_duration_s":1.16902,"scheduling_latency_s":1.169039,"redis_calls":13,"redis_duration_s":0.002722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001781,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017494,"completed_at":"2026-06-14T14:28:39.457Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462vm-test-run-nixbot-gitlab> gitlab # [  227.606729] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4b9d9076a8a4858885cec85","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4b9d9076a8a4858885cec85: start","job_status":"start","queue_duration_s":1.188067,"scheduling_latency_s":1.188085}1463vm-test-run-nixbot-gitlab> gitlab # [  227.618407] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4b9d9076a8a4858885cec85","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4b9d9076a8a4858885cec85: done: 0.016156 sec","job_status":"done","queue_duration_s":1.188067,"scheduling_latency_s":1.188085,"redis_calls":13,"redis_duration_s":0.002754,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016156,"completed_at":"2026-06-14T14:28:39.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464vm-test-run-nixbot-gitlab> gitlab # [  227.655159] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90c244bc5390db2063bcf2e4","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90c244bc5390db2063bcf2e4: start","job_status":"start","queue_duration_s":1.205651,"scheduling_latency_s":1.205667}1465vm-test-run-nixbot-gitlab> gitlab # [  227.666806] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90c244bc5390db2063bcf2e4","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90c244bc5390db2063bcf2e4: done: 0.015454 sec","job_status":"done","queue_duration_s":1.205651,"scheduling_latency_s":1.205667,"redis_calls":13,"redis_duration_s":0.002821,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015454,"completed_at":"2026-06-14T14:28:39.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [  227.704193] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7de6e2647c069dcb4c6c706f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7de6e2647c069dcb4c6c706f: start","job_status":"start","queue_duration_s":1.223474,"scheduling_latency_s":1.223647}1467vm-test-run-nixbot-gitlab> gitlab # [  227.715599] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7de6e2647c069dcb4c6c706f","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7de6e2647c069dcb4c6c706f: done: 0.015983 sec","job_status":"done","queue_duration_s":1.223474,"scheduling_latency_s":1.223647,"redis_calls":13,"redis_duration_s":0.0025599999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001581,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015983,"completed_at":"2026-06-14T14:28:39.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [  227.752369] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61398085cf5cea1deaa81d6d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61398085cf5cea1deaa81d6d: start","job_status":"start","queue_duration_s":1.240903,"scheduling_latency_s":1.240919}1469vm-test-run-nixbot-gitlab> gitlab # [  227.763927] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61398085cf5cea1deaa81d6d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61398085cf5cea1deaa81d6d: done: 0.015322 sec","job_status":"done","queue_duration_s":1.240903,"scheduling_latency_s":1.240919,"redis_calls":13,"redis_duration_s":0.0025789999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001661,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015322,"completed_at":"2026-06-14T14:28:39.526Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [  227.800570] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b78ea84e45cbcb54d7d15cc0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b78ea84e45cbcb54d7d15cc0: start","job_status":"start","queue_duration_s":1.257734,"scheduling_latency_s":1.25775}1471vm-test-run-nixbot-gitlab> gitlab # [  227.812048] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b78ea84e45cbcb54d7d15cc0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b78ea84e45cbcb54d7d15cc0: done: 0.015201 sec","job_status":"done","queue_duration_s":1.257734,"scheduling_latency_s":1.25775,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015201,"completed_at":"2026-06-14T14:28:39.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [  227.848318] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d5f810dc60fa79c4db771db","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d5f810dc60fa79c4db771db: start","job_status":"start","queue_duration_s":1.274276,"scheduling_latency_s":1.274292}1473vm-test-run-nixbot-gitlab> gitlab # [  227.859793] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d5f810dc60fa79c4db771db","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d5f810dc60fa79c4db771db: done: 0.015455 sec","job_status":"done","queue_duration_s":1.274276,"scheduling_latency_s":1.274292,"redis_calls":13,"redis_duration_s":0.002749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015455,"completed_at":"2026-06-14T14:28:39.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474vm-test-run-nixbot-gitlab> gitlab # [  227.896758] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e47b686d724ec3fc87c806d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e47b686d724ec3fc87c806d: start","job_status":"start","queue_duration_s":1.290992,"scheduling_latency_s":1.291009}1475vm-test-run-nixbot-gitlab> gitlab # [  227.909179] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e47b686d724ec3fc87c806d","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e47b686d724ec3fc87c806d: done: 0.015861 sec","job_status":"done","queue_duration_s":1.290992,"scheduling_latency_s":1.291009,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015861,"completed_at":"2026-06-14T14:28:39.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [  227.945191] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8269d1903cb236382b861bb0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8269d1903cb236382b861bb0: start","job_status":"start","queue_duration_s":1.308241,"scheduling_latency_s":1.308258}1477vm-test-run-nixbot-gitlab> gitlab # [  227.956757] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8269d1903cb236382b861bb0","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8269d1903cb236382b861bb0: done: 0.015613 sec","job_status":"done","queue_duration_s":1.308241,"scheduling_latency_s":1.308258,"redis_calls":13,"redis_duration_s":0.002755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015613,"completed_at":"2026-06-14T14:28:39.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [  227.992813] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb4f060da75f8cb6773f6e35","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb4f060da75f8cb6773f6e35: start","job_status":"start","queue_duration_s":1.32519,"scheduling_latency_s":1.325207}1479vm-test-run-nixbot-gitlab> gitlab # [  228.004222] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb4f060da75f8cb6773f6e35","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb4f060da75f8cb6773f6e35: done: 0.015892 sec","job_status":"done","queue_duration_s":1.32519,"scheduling_latency_s":1.325207,"redis_calls":13,"redis_duration_s":0.002694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015892,"completed_at":"2026-06-14T14:28:39.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [  228.040187] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"585e6115c3bb94dd22a743ce","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-585e6115c3bb94dd22a743ce: start","job_status":"start","queue_duration_s":1.342373,"scheduling_latency_s":1.342389}1481vm-test-run-nixbot-gitlab> gitlab # [  228.051666] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"585e6115c3bb94dd22a743ce","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-585e6115c3bb94dd22a743ce: done: 0.015722 sec","job_status":"done","queue_duration_s":1.342373,"scheduling_latency_s":1.342389,"redis_calls":13,"redis_duration_s":0.002772,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001801,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015722,"completed_at":"2026-06-14T14:28:39.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [  228.087891] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc43c96a39ce4b19bac541a3","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc43c96a39ce4b19bac541a3: start","job_status":"start","queue_duration_s":1.359636,"scheduling_latency_s":1.359652}1483vm-test-run-nixbot-gitlab> gitlab # [  228.099622] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc43c96a39ce4b19bac541a3","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc43c96a39ce4b19bac541a3: done: 0.015576 sec","job_status":"done","queue_duration_s":1.359636,"scheduling_latency_s":1.359652,"redis_calls":13,"redis_duration_s":0.0026550000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001729,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-06-14T14:28:39.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [  228.136949] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2d78728256fcf8c2d3d39c","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2d78728256fcf8c2d3d39c: start","job_status":"start","queue_duration_s":1.376701,"scheduling_latency_s":1.376718}1485vm-test-run-nixbot-gitlab> gitlab # [  228.148258] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2d78728256fcf8c2d3d39c","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2d78728256fcf8c2d3d39c: done: 0.015307 sec","job_status":"done","queue_duration_s":1.376701,"scheduling_latency_s":1.376718,"redis_calls":13,"redis_duration_s":0.00266,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001763,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015307,"completed_at":"2026-06-14T14:28:39.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [  228.184380] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b64a3f09d76753b31b21840e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b64a3f09d76753b31b21840e: start","job_status":"start","queue_duration_s":1.394213,"scheduling_latency_s":1.394231}1487vm-test-run-nixbot-gitlab> gitlab # [  228.195835] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b64a3f09d76753b31b21840e","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b64a3f09d76753b31b21840e: done: 0.01542 sec","job_status":"done","queue_duration_s":1.394213,"scheduling_latency_s":1.394231,"redis_calls":13,"redis_duration_s":0.002675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01542,"completed_at":"2026-06-14T14:28:39.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [  228.232462] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d01fb31a2002cf16508d6176","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d01fb31a2002cf16508d6176: start","job_status":"start","queue_duration_s":1.411181,"scheduling_latency_s":1.411199}1489vm-test-run-nixbot-gitlab> gitlab # [  228.243863] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d01fb31a2002cf16508d6176","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d01fb31a2002cf16508d6176: done: 0.01588 sec","job_status":"done","queue_duration_s":1.411181,"scheduling_latency_s":1.411199,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01588,"completed_at":"2026-06-14T14:28:39.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490vm-test-run-nixbot-gitlab> gitlab # [  228.280276] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45ce4d32ef6d7ccda33e544","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45ce4d32ef6d7ccda33e544: start","job_status":"start","queue_duration_s":1.428621,"scheduling_latency_s":1.428638}1491vm-test-run-nixbot-gitlab> gitlab # [  228.291703] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45ce4d32ef6d7ccda33e544","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45ce4d32ef6d7ccda33e544: done: 0.015394 sec","job_status":"done","queue_duration_s":1.428621,"scheduling_latency_s":1.428638,"redis_calls":13,"redis_duration_s":0.002633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015394,"completed_at":"2026-06-14T14:28:39.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492vm-test-run-nixbot-gitlab> gitlab # [  228.329189] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ad407f2a3d989cd9cc3b401","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ad407f2a3d989cd9cc3b401: start","job_status":"start","queue_duration_s":1.445354,"scheduling_latency_s":1.445371}1493vm-test-run-nixbot-gitlab> gitlab # [  228.340786] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ad407f2a3d989cd9cc3b401","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ad407f2a3d989cd9cc3b401: done: 0.135944 sec","job_status":"done","queue_duration_s":1.445354,"scheduling_latency_s":1.445371,"redis_calls":13,"redis_duration_s":0.002749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.135944,"completed_at":"2026-06-14T14:28:39.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494vm-test-run-nixbot-gitlab> gitlab # [  228.377255] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8690ef1fe896acc1bacccc8b","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8690ef1fe896acc1bacccc8b: start","job_status":"start","queue_duration_s":1.582892,"scheduling_latency_s":1.582912}1495vm-test-run-nixbot-gitlab> gitlab # [  228.388792] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:28:39.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8690ef1fe896acc1bacccc8b","created_at":"2026-06-14T14:28:38.165Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-d871b9642443478f","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3157fc2ce664b4e8d7f554780639d366","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:28:38.270Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8690ef1fe896acc1bacccc8b: done: 0.016542 sec","job_status":"done","queue_duration_s":1.582892,"scheduling_latency_s":1.582912,"redis_calls":13,"redis_duration_s":0.002786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016542,"completed_at":"2026-06-14T14:28:39.870Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496vm-test-run-nixbot-gitlab> gitlab # [  251.628487] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:04.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7e1ab06811a4a01f212152d2","created_at":"2026-06-14T14:29:04.511Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c243c9a9664ff48fdba25fc049d451e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.523Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7e1ab06811a4a01f212152d2: start","job_status":"start","queue_duration_s":0.003324,"scheduling_latency_s":0.003392}1497vm-test-run-nixbot-gitlab> gitlab # [  251.751090] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:04.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7e1ab06811a4a01f212152d2","created_at":"2026-06-14T14:29:04.511Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c243c9a9664ff48fdba25fc049d451e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.523Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7e1ab06811a4a01f212152d2: done: 0.123776 sec","job_status":"done","queue_duration_s":0.003324,"scheduling_latency_s":0.003392,"redis_calls":8,"redis_duration_s":0.015578000000000002,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001541,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002942,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.123776,"completed_at":"2026-06-14T14:29:04.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [  251.844849] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:04.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"78a8d48fede90f0a28d52e86","created_at":"2026-06-14T14:29:04.548Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.561Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-78a8d48fede90f0a28d52e86: start","job_status":"start","queue_duration_s":0.15479,"scheduling_latency_s":0.154854}1499vm-test-run-nixbot-gitlab> gitlab # [  252.023992] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:04.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"78a8d48fede90f0a28d52e86","created_at":"2026-06-14T14:29:04.548Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.561Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-78a8d48fede90f0a28d52e86: done: 0.207724 sec","job_status":"done","queue_duration_s":0.15479,"scheduling_latency_s":0.154854,"redis_calls":14,"redis_duration_s":0.057323,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.001146,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.039455,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.016722,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.207724,"completed_at":"2026-06-14T14:29:04.923Z","load_balancing_strategy":"primary","db_duration_s":0.010189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [  252.116390] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:04.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fe826d75d2fe7567fc9cc2a4","created_at":"2026-06-14T14:29:04.631Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74d2e8bf03909a1ac0a29542f25dcc33","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.642Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fe826d75d2fe7567fc9cc2a4: start","job_status":"start","queue_duration_s":0.326151,"scheduling_latency_s":0.326193}1501vm-test-run-nixbot-gitlab> gitlab # [  252.151781] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fe826d75d2fe7567fc9cc2a4","created_at":"2026-06-14T14:29:04.631Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74d2e8bf03909a1ac0a29542f25dcc33","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.642Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fe826d75d2fe7567fc9cc2a4: done: 0.083508 sec","job_status":"done","queue_duration_s":0.326151,"scheduling_latency_s":0.326193,"redis_calls":9,"redis_duration_s":0.006033,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00081384,"exclusive_lock_hold_duration_s":0.02295814,"duration_s":0.083508,"completed_at":"2026-06-14T14:29:05.052Z","load_balancing_strategy":"primary","db_duration_s":0.019938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [  252.214960] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1d652fd959f6fc4665989649","created_at":"2026-06-14T14:29:04.709Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb0a0571a792681a2c40b9e989f40c3b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.719Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1d652fd959f6fc4665989649: start","job_status":"start","queue_duration_s":0.334346,"scheduling_latency_s":0.334371}1503vm-test-run-nixbot-gitlab> gitlab # [  252.233491] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1d652fd959f6fc4665989649","created_at":"2026-06-14T14:29:04.709Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb0a0571a792681a2c40b9e989f40c3b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.719Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1d652fd959f6fc4665989649: done: 0.01903 sec","job_status":"done","queue_duration_s":0.334346,"scheduling_latency_s":0.334371,"redis_calls":9,"redis_duration_s":0.001484,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058734,"exclusive_lock_hold_duration_s":0.0033177,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01903,"completed_at":"2026-06-14T14:29:05.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [  252.275306] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.073Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"17ee73b3871e3fffe61cbc82","created_at":"2026-06-14T14:29:04.751Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecf05777c6e788c3c5c76235d2d0e9b4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.755Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-17ee73b3871e3fffe61cbc82: start","job_status":"start","queue_duration_s":0.318731,"scheduling_latency_s":0.318753}1505vm-test-run-nixbot-gitlab> gitlab # [  252.288593] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.099Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"17ee73b3871e3fffe61cbc82","created_at":"2026-06-14T14:29:04.751Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecf05777c6e788c3c5c76235d2d0e9b4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.755Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-17ee73b3871e3fffe61cbc82: done: 0.025515 sec","job_status":"done","queue_duration_s":0.318731,"scheduling_latency_s":0.318753,"redis_calls":7,"redis_duration_s":0.001073,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025515,"completed_at":"2026-06-14T14:29:05.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [  252.326195] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"33edac80ca264d2f1a07f55b","created_at":"2026-06-14T14:29:04.759Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d34377bb608bbb28fa6d6382fedf49a7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.762Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-33edac80ca264d2f1a07f55b: start","job_status":"start","queue_duration_s":0.337967,"scheduling_latency_s":0.337987}1507vm-test-run-nixbot-gitlab> gitlab # [  252.493178] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"33edac80ca264d2f1a07f55b","created_at":"2026-06-14T14:29:04.759Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d34377bb608bbb28fa6d6382fedf49a7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.762Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-33edac80ca264d2f1a07f55b: done: 0.292816 sec","job_status":"done","queue_duration_s":0.337967,"scheduling_latency_s":0.337987,"redis_calls":571,"redis_duration_s":0.054773,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.054051,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.2728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.292816,"completed_at":"2026-06-14T14:29:05.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [  252.574362] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1d7f4f54a251186aa3fbd072","created_at":"2026-06-14T14:29:04.771Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e9823eaafa7bab2a04376cfe803d382","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.776Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1d7f4f54a251186aa3fbd072: start","job_status":"start","queue_duration_s":0.61793,"scheduling_latency_s":0.617952}1509vm-test-run-nixbot-gitlab> gitlab # [  252.591831] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1d7f4f54a251186aa3fbd072","created_at":"2026-06-14T14:29:04.771Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e9823eaafa7bab2a04376cfe803d382","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.776Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1d7f4f54a251186aa3fbd072: done: 0.039547 sec","job_status":"done","queue_duration_s":0.61793,"scheduling_latency_s":0.617952,"redis_calls":9,"redis_duration_s":0.001934,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00084418,"exclusive_lock_hold_duration_s":0.01616474,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.039547,"completed_at":"2026-06-14T14:29:05.434Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510vm-test-run-nixbot-gitlab> gitlab # [  252.635395] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"910d7a0a9761502e9294e267","created_at":"2026-06-14T14:29:04.780Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5789a47212f13beddbbebe302eb13406","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.791Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-910d7a0a9761502e9294e267: start","job_status":"start","queue_duration_s":0.644234,"scheduling_latency_s":0.644254}1511vm-test-run-nixbot-gitlab> gitlab # [  252.648866] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"910d7a0a9761502e9294e267","created_at":"2026-06-14T14:29:04.780Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5789a47212f13beddbbebe302eb13406","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.791Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-910d7a0a9761502e9294e267: done: 0.018175 sec","job_status":"done","queue_duration_s":0.644234,"scheduling_latency_s":0.644254,"redis_calls":7,"redis_duration_s":0.00144,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018175,"completed_at":"2026-06-14T14:29:05.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [  252.686158] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e780f262faf0513431a473cd","created_at":"2026-06-14T14:29:04.801Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3262cf70a6021513af2feff8e1e7068c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.805Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e780f262faf0513431a473cd: start","job_status":"start","queue_duration_s":0.65073,"scheduling_latency_s":0.650754}1513vm-test-run-nixbot-gitlab> gitlab # [  252.699861] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e780f262faf0513431a473cd","created_at":"2026-06-14T14:29:04.801Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3262cf70a6021513af2feff8e1e7068c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.805Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e780f262faf0513431a473cd: done: 0.077281 sec","job_status":"done","queue_duration_s":0.65073,"scheduling_latency_s":0.650754,"redis_calls":9,"redis_duration_s":0.002351,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006838,"exclusive_lock_hold_duration_s":0.05942594,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.077281,"completed_at":"2026-06-14T14:29:05.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514vm-test-run-nixbot-gitlab> gitlab # [  252.740581] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"51a397d196e5037ee77a7acb","created_at":"2026-06-14T14:29:04.809Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb0bdce4df78d2891587c488b4c8c146","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.815Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-51a397d196e5037ee77a7acb: start","job_status":"start","queue_duration_s":0.719437,"scheduling_latency_s":0.719464}1515vm-test-run-nixbot-gitlab> gitlab # [  252.939150] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"51a397d196e5037ee77a7acb","created_at":"2026-06-14T14:29:04.809Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb0bdce4df78d2891587c488b4c8c146","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.815Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-51a397d196e5037ee77a7acb: done: 0.303607 sec","job_status":"done","queue_duration_s":0.719437,"scheduling_latency_s":0.719464,"redis_calls":9,"redis_duration_s":0.002737,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.240909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069074,"exclusive_lock_hold_duration_s":0.28126016,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.303607,"completed_at":"2026-06-14T14:29:05.838Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [  253.011939] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ab4e96050e5519386da48f37","created_at":"2026-06-14T14:29:04.819Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c01913e345f2af623ef3fbe6ba97a64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T14:29:04.823Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ab4e96050e5519386da48f37: start","job_status":"start","queue_duration_s":1.018731,"scheduling_latency_s":1.01878}1517vm-test-run-nixbot-gitlab> gitlab # [  253.027542] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.861Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1518vm-test-run-nixbot-gitlab> gitlab # [  253.031220] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.873Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ab4e96050e5519386da48f37","created_at":"2026-06-14T14:29:04.819Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c01913e345f2af623ef3fbe6ba97a64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T14:29:04.823Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ab4e96050e5519386da48f37: done: 0.031584 sec","job_status":"done","queue_duration_s":1.018731,"scheduling_latency_s":1.01878,"redis_calls":7,"redis_duration_s":0.002899,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031584,"completed_at":"2026-06-14T14:29:05.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [  253.073068] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fe6bcdf6e822d2dd81a9b8f0","created_at":"2026-06-14T14:29:04.835Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b32f9c7cb1e5df9d4b5cffc793a3112","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.844Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fe6bcdf6e822d2dd81a9b8f0: start","job_status":"start","queue_duration_s":1.032457,"scheduling_latency_s":1.032514}1520vm-test-run-nixbot-gitlab> gitlab # [  253.087265] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fe6bcdf6e822d2dd81a9b8f0","created_at":"2026-06-14T14:29:04.835Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b32f9c7cb1e5df9d4b5cffc793a3112","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.844Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fe6bcdf6e822d2dd81a9b8f0: done: 0.023264 sec","job_status":"done","queue_duration_s":1.032457,"scheduling_latency_s":1.032514,"redis_calls":7,"redis_duration_s":0.002085,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001499,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023264,"completed_at":"2026-06-14T14:29:05.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [  253.125422] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:05.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"85f38fc71cd7eec0c3a2f685","created_at":"2026-06-14T14:29:04.849Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.854Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-85f38fc71cd7eec0c3a2f685: start","job_status":"start","queue_duration_s":1.047456,"scheduling_latency_s":1.047484}1522vm-test-run-nixbot-gitlab> gitlab # [  253.138119] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"85f38fc71cd7eec0c3a2f685","created_at":"2026-06-14T14:29:04.849Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.854Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-85f38fc71cd7eec0c3a2f685: done: 0.099201 sec","job_status":"done","queue_duration_s":1.047456,"scheduling_latency_s":1.047484,"redis_calls":60,"redis_duration_s":0.015345000000000001,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000721,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013823,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099201,"completed_at":"2026-06-14T14:29:06.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [  253.176550] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"84b2c80cf48e224141c95f4e","created_at":"2026-06-14T14:29:04.882Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"582966b798902c98d098c6d7a051c041","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.890Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-84b2c80cf48e224141c95f4e: start","job_status":"start","queue_duration_s":1.112165,"scheduling_latency_s":1.112186}1524vm-test-run-nixbot-gitlab> gitlab # [  253.189969] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.010Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1525vm-test-run-nixbot-gitlab> gitlab # [  253.192956] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"84b2c80cf48e224141c95f4e","created_at":"2026-06-14T14:29:04.882Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"582966b798902c98d098c6d7a051c041","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.890Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-84b2c80cf48e224141c95f4e: done: 0.013996 sec","job_status":"done","queue_duration_s":1.112165,"scheduling_latency_s":1.112186,"redis_calls":7,"redis_duration_s":0.001262,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013996,"completed_at":"2026-06-14T14:29:06.016Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [  253.231568] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.017Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c840b2cad0722231e88df02f","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c840b2cad0722231e88df02f: start","job_status":"start","queue_duration_s":1.126289,"scheduling_latency_s":1.12631}1527vm-test-run-nixbot-gitlab> gitlab # [  253.243164] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.029Z","message":"No existing merge request to be cleaned up."}1528vm-test-run-nixbot-gitlab> gitlab # [  253.244775] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.038Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c840b2cad0722231e88df02f","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c840b2cad0722231e88df02f: done: 0.020773 sec","job_status":"done","queue_duration_s":1.126289,"scheduling_latency_s":1.12631,"redis_calls":10,"redis_duration_s":0.001719,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020773,"completed_at":"2026-06-14T14:29:06.038Z","load_balancing_strategy":"primary","db_duration_s":0.002119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529vm-test-run-nixbot-gitlab> gitlab # [  253.280757] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.039Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ad0786ffe8a6bd19b0b5b294","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ad0786ffe8a6bd19b0b5b294: start","job_status":"start","queue_duration_s":1.148471,"scheduling_latency_s":1.148495}1530vm-test-run-nixbot-gitlab> gitlab # [  253.292116] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.050Z","message":"No existing merge request to be cleaned up."}1531vm-test-run-nixbot-gitlab> gitlab # [  253.293631] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.058Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ad0786ffe8a6bd19b0b5b294","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ad0786ffe8a6bd19b0b5b294: done: 0.01885 sec","job_status":"done","queue_duration_s":1.148471,"scheduling_latency_s":1.148495,"redis_calls":10,"redis_duration_s":0.001438,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000833,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01885,"completed_at":"2026-06-14T14:29:06.058Z","load_balancing_strategy":"primary","db_duration_s":0.001729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [  253.329665] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.059Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea44e14c3b46801e38c89812","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea44e14c3b46801e38c89812: start","job_status":"start","queue_duration_s":1.168474,"scheduling_latency_s":1.168492}1533vm-test-run-nixbot-gitlab> gitlab # [  253.341195] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.069Z","message":"No existing merge request to be cleaned up."}1534vm-test-run-nixbot-gitlab> gitlab # [  253.342727] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.078Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea44e14c3b46801e38c89812","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea44e14c3b46801e38c89812: done: 0.018726 sec","job_status":"done","queue_duration_s":1.168474,"scheduling_latency_s":1.168492,"redis_calls":10,"redis_duration_s":0.0014,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018726,"completed_at":"2026-06-14T14:29:06.078Z","load_balancing_strategy":"primary","db_duration_s":0.002149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535vm-test-run-nixbot-gitlab> gitlab # [  253.378723] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.079Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8d10915aae4d984c70d958ac","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8d10915aae4d984c70d958ac: start","job_status":"start","queue_duration_s":1.188279,"scheduling_latency_s":1.188297}1536vm-test-run-nixbot-gitlab> gitlab # [  253.390400] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.089Z","message":"No existing merge request to be cleaned up."}1537vm-test-run-nixbot-gitlab> gitlab # [  253.391938] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.097Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8d10915aae4d984c70d958ac","created_at":"2026-06-14T14:29:04.855Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-7194903f61524627","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"599bfed11fd59589f2b514599a2ff877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.891Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8d10915aae4d984c70d958ac: done: 0.018019 sec","job_status":"done","queue_duration_s":1.188279,"scheduling_latency_s":1.188297,"redis_calls":10,"redis_duration_s":0.001354,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018019,"completed_at":"2026-06-14T14:29:06.097Z","load_balancing_strategy":"primary","db_duration_s":0.001702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [  253.427876] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a315a96f8adb39adf64f5cd7","created_at":"2026-06-14T14:29:04.905Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f15ab4fa5f1496c6122e54f6ecbd5df2","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.917Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a315a96f8adb39adf64f5cd7: start","job_status":"start","queue_duration_s":1.180924,"scheduling_latency_s":1.180942}1539vm-test-run-nixbot-gitlab> gitlab # [  253.442481] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a315a96f8adb39adf64f5cd7","created_at":"2026-06-14T14:29:04.905Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f15ab4fa5f1496c6122e54f6ecbd5df2","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.917Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a315a96f8adb39adf64f5cd7: done: 0.108688 sec","job_status":"done","queue_duration_s":1.180924,"scheduling_latency_s":1.180942,"redis_calls":7,"redis_duration_s":0.001106,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.108688,"completed_at":"2026-06-14T14:29:06.207Z","load_balancing_strategy":"primary","db_duration_s":0.007148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [  253.480785] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"656089412a5ae521766a8101","created_at":"2026-06-14T14:29:04.964Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36a610c9b3b76fd070eefb09b67ee2c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.972Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-656089412a5ae521766a8101: start","job_status":"start","queue_duration_s":1.236318,"scheduling_latency_s":1.236337}1541vm-test-run-nixbot-gitlab> gitlab # [  253.494377] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"656089412a5ae521766a8101","created_at":"2026-06-14T14:29:04.964Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36a610c9b3b76fd070eefb09b67ee2c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:04.972Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-656089412a5ae521766a8101: done: 0.021278 sec","job_status":"done","queue_duration_s":1.236318,"scheduling_latency_s":1.236337,"redis_calls":10,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00491852,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021278,"completed_at":"2026-06-14T14:29:06.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002504,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1542vm-test-run-nixbot-gitlab> gitlab # [  253.534230] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a6097200bab70a438c9d1d70","created_at":"2026-06-14T14:29:04.985Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf1d511a8c97e8e5ddd0f4efc1b57b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T14:29:05.007Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a6097200bab70a438c9d1d70: start","job_status":"start","queue_duration_s":1.223134,"scheduling_latency_s":1.223153}1543vm-test-run-nixbot-gitlab> gitlab # [  253.546575] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a6097200bab70a438c9d1d70","created_at":"2026-06-14T14:29:04.985Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf1d511a8c97e8e5ddd0f4efc1b57b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T14:29:05.007Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a6097200bab70a438c9d1d70: done: 0.021899 sec","job_status":"done","queue_duration_s":1.223134,"scheduling_latency_s":1.223153,"redis_calls":7,"redis_duration_s":0.001094,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021899,"completed_at":"2026-06-14T14:29:06.252Z","load_balancing_strategy":"primary","db_duration_s":0.001663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [  253.583209] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1f118a67bc805e78cf274df4","created_at":"2026-06-14T14:29:05.013Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b709922945845246afa18f717046e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T14:29:05.017Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1f118a67bc805e78cf274df4: start","job_status":"start","queue_duration_s":1.237446,"scheduling_latency_s":1.237464}1545vm-test-run-nixbot-gitlab> gitlab # [  253.595873] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.262Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1546vm-test-run-nixbot-gitlab> gitlab # [  253.599542] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1f118a67bc805e78cf274df4","created_at":"2026-06-14T14:29:05.013Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b709922945845246afa18f717046e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T14:29:05.017Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1f118a67bc805e78cf274df4: done: 0.013347 sec","job_status":"done","queue_duration_s":1.237446,"scheduling_latency_s":1.237464,"redis_calls":7,"redis_duration_s":0.000976,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013347,"completed_at":"2026-06-14T14:29:06.268Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [  253.637634] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a85972dcabf12bb2ce8affbe","created_at":"2026-06-14T14:29:05.020Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"216e9baa4e44ee9133ebe7f0566a5356","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.025Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a85972dcabf12bb2ce8affbe: start","job_status":"start","queue_duration_s":1.244179,"scheduling_latency_s":1.244197}1548vm-test-run-nixbot-gitlab> gitlab # [  253.651243] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.276Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1549vm-test-run-nixbot-gitlab> gitlab # [  253.654547] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a85972dcabf12bb2ce8affbe","created_at":"2026-06-14T14:29:05.020Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"216e9baa4e44ee9133ebe7f0566a5356","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.025Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a85972dcabf12bb2ce8affbe: done: 0.013734 sec","job_status":"done","queue_duration_s":1.244179,"scheduling_latency_s":1.244197,"redis_calls":7,"redis_duration_s":0.000994,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013734,"completed_at":"2026-06-14T14:29:06.282Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [  253.693029] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"cc220360716c3da869dd1b7a","created_at":"2026-06-14T14:29:05.027Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17471b1c6c30be2e179fde608a4cecec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.032Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-cc220360716c3da869dd1b7a: start","job_status":"start","queue_duration_s":1.251673,"scheduling_latency_s":1.25169}1551vm-test-run-nixbot-gitlab> gitlab # [  253.706360] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.291Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1552vm-test-run-nixbot-gitlab> gitlab # [  253.709943] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"cc220360716c3da869dd1b7a","created_at":"2026-06-14T14:29:05.027Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17471b1c6c30be2e179fde608a4cecec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.032Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-cc220360716c3da869dd1b7a: done: 0.013145 sec","job_status":"done","queue_duration_s":1.251673,"scheduling_latency_s":1.25169,"redis_calls":7,"redis_duration_s":0.0010140000000000001,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013145,"completed_at":"2026-06-14T14:29:06.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [  253.748248] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dd83f52a0eafe06b1be6fbfb","created_at":"2026-06-14T14:29:05.036Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4982a802fb6c396938e47e6c8e4c1fdf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.041Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dd83f52a0eafe06b1be6fbfb: start","job_status":"start","queue_duration_s":1.256604,"scheduling_latency_s":1.256621}1554vm-test-run-nixbot-gitlab> gitlab # [  253.761666] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.307Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1555vm-test-run-nixbot-gitlab> gitlab # [  253.765055] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dd83f52a0eafe06b1be6fbfb","created_at":"2026-06-14T14:29:05.036Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4982a802fb6c396938e47e6c8e4c1fdf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.041Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dd83f52a0eafe06b1be6fbfb: done: 0.014 sec","job_status":"done","queue_duration_s":1.256604,"scheduling_latency_s":1.256621,"redis_calls":7,"redis_duration_s":0.002583,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002312,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014,"completed_at":"2026-06-14T14:29:06.312Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [  253.802619] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"443973aa148410be8973ece6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-443973aa148410be8973ece6: start","job_status":"start","queue_duration_s":0.320025,"scheduling_latency_s":0.320042}1557vm-test-run-nixbot-gitlab> gitlab # [  253.814313] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"443973aa148410be8973ece6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-443973aa148410be8973ece6: done: 0.015299 sec","job_status":"done","queue_duration_s":0.320025,"scheduling_latency_s":0.320042,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015299,"completed_at":"2026-06-14T14:29:06.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [  253.852047] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44c4eb2cd4c0bd8de3c9294e","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44c4eb2cd4c0bd8de3c9294e: start","job_status":"start","queue_duration_s":0.33739,"scheduling_latency_s":0.337408}1559vm-test-run-nixbot-gitlab> gitlab # [  253.863469] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44c4eb2cd4c0bd8de3c9294e","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44c4eb2cd4c0bd8de3c9294e: done: 0.015241 sec","job_status":"done","queue_duration_s":0.33739,"scheduling_latency_s":0.337408,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015241,"completed_at":"2026-06-14T14:29:06.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [  253.901141] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9283487e8974780d5ad0536f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9283487e8974780d5ad0536f: start","job_status":"start","queue_duration_s":0.353706,"scheduling_latency_s":0.353723}1561vm-test-run-nixbot-gitlab> gitlab # [  253.912927] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9283487e8974780d5ad0536f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9283487e8974780d5ad0536f: done: 0.014896 sec","job_status":"done","queue_duration_s":0.353706,"scheduling_latency_s":0.353723,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014896,"completed_at":"2026-06-14T14:29:06.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [  253.949780] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd1e29c0e12669be37d094f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd1e29c0e12669be37d094f: start","job_status":"start","queue_duration_s":0.369652,"scheduling_latency_s":0.369669}1563vm-test-run-nixbot-gitlab> gitlab # [  253.961646] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd1e29c0e12669be37d094f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd1e29c0e12669be37d094f: done: 0.015227 sec","job_status":"done","queue_duration_s":0.369652,"scheduling_latency_s":0.369669,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015227,"completed_at":"2026-06-14T14:29:06.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [  253.998766] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a63fc205d0b754d0fcdf6d3","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a63fc205d0b754d0fcdf6d3: start","job_status":"start","queue_duration_s":0.385925,"scheduling_latency_s":0.385942}1565vm-test-run-nixbot-gitlab> gitlab # [  254.010526] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a63fc205d0b754d0fcdf6d3","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a63fc205d0b754d0fcdf6d3: done: 0.015706 sec","job_status":"done","queue_duration_s":0.385925,"scheduling_latency_s":0.385942,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015706,"completed_at":"2026-06-14T14:29:06.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [  254.048404] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d593dbd8eb51be428e5be01","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d593dbd8eb51be428e5be01: start","job_status":"start","queue_duration_s":0.402737,"scheduling_latency_s":0.402754}1567vm-test-run-nixbot-gitlab> gitlab # [  254.059922] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d593dbd8eb51be428e5be01","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d593dbd8eb51be428e5be01: done: 0.015092 sec","job_status":"done","queue_duration_s":0.402737,"scheduling_latency_s":0.402754,"redis_calls":13,"redis_duration_s":0.001554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015092,"completed_at":"2026-06-14T14:29:06.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568vm-test-run-nixbot-gitlab> gitlab # [  254.097263] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a3f3d4e106d0c0941d60e4","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a3f3d4e106d0c0941d60e4: start","job_status":"start","queue_duration_s":0.418867,"scheduling_latency_s":0.418884}1569vm-test-run-nixbot-gitlab> gitlab # [  254.109202] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a3f3d4e106d0c0941d60e4","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a3f3d4e106d0c0941d60e4: done: 0.014984 sec","job_status":"done","queue_duration_s":0.418867,"scheduling_latency_s":0.418884,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014984,"completed_at":"2026-06-14T14:29:06.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [  254.145472] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b89a5698fd40566517ea078f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b89a5698fd40566517ea078f: start","job_status":"start","queue_duration_s":0.434924,"scheduling_latency_s":0.43494}1571vm-test-run-nixbot-gitlab> gitlab # [  254.157016] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b89a5698fd40566517ea078f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b89a5698fd40566517ea078f: done: 0.015086 sec","job_status":"done","queue_duration_s":0.434924,"scheduling_latency_s":0.43494,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-06-14T14:29:06.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [  254.193407] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983fcb590c69483af9fe36dd","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983fcb590c69483af9fe36dd: start","job_status":"start","queue_duration_s":0.452053,"scheduling_latency_s":0.452071}1573vm-test-run-nixbot-gitlab> gitlab # [  254.204956] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983fcb590c69483af9fe36dd","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983fcb590c69483af9fe36dd: done: 0.015517 sec","job_status":"done","queue_duration_s":0.452053,"scheduling_latency_s":0.452071,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015517,"completed_at":"2026-06-14T14:29:06.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574vm-test-run-nixbot-gitlab> gitlab # [  254.241647] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757d031ca9af116d172d06d8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757d031ca9af116d172d06d8: start","job_status":"start","queue_duration_s":0.468682,"scheduling_latency_s":0.468699}1575vm-test-run-nixbot-gitlab> gitlab # [  254.253978] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757d031ca9af116d172d06d8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757d031ca9af116d172d06d8: done: 0.015645 sec","job_status":"done","queue_duration_s":0.468682,"scheduling_latency_s":0.468699,"redis_calls":13,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015645,"completed_at":"2026-06-14T14:29:06.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [  254.290458] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a40a6dc400af3f88317f1e37","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a40a6dc400af3f88317f1e37: start","job_status":"start","queue_duration_s":0.485441,"scheduling_latency_s":0.485458}1577vm-test-run-nixbot-gitlab> gitlab # [  254.302086] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a40a6dc400af3f88317f1e37","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a40a6dc400af3f88317f1e37: done: 0.01519 sec","job_status":"done","queue_duration_s":0.485441,"scheduling_latency_s":0.485458,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01519,"completed_at":"2026-06-14T14:29:06.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [  254.338579] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de0f4bd1a989d7e0b90afda1","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de0f4bd1a989d7e0b90afda1: start","job_status":"start","queue_duration_s":0.501712,"scheduling_latency_s":0.501728}1579vm-test-run-nixbot-gitlab> gitlab # [  254.350234] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de0f4bd1a989d7e0b90afda1","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de0f4bd1a989d7e0b90afda1: done: 0.016347 sec","job_status":"done","queue_duration_s":0.501712,"scheduling_latency_s":0.501728,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016347,"completed_at":"2026-06-14T14:29:06.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [  254.386464] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be7b5fc0d046a34b28d48d90","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be7b5fc0d046a34b28d48d90: start","job_status":"start","queue_duration_s":0.5196,"scheduling_latency_s":0.519619}1581vm-test-run-nixbot-gitlab> gitlab # [  254.398022] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be7b5fc0d046a34b28d48d90","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be7b5fc0d046a34b28d48d90: done: 0.016395 sec","job_status":"done","queue_duration_s":0.5196,"scheduling_latency_s":0.519619,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016395,"completed_at":"2026-06-14T14:29:06.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [  254.434518] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5017ca870abde5cb16d0bde2","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5017ca870abde5cb16d0bde2: start","job_status":"start","queue_duration_s":0.537018,"scheduling_latency_s":0.537035}1583vm-test-run-nixbot-gitlab> gitlab # [  254.446194] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5017ca870abde5cb16d0bde2","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5017ca870abde5cb16d0bde2: done: 0.015433 sec","job_status":"done","queue_duration_s":0.537018,"scheduling_latency_s":0.537035,"redis_calls":13,"redis_duration_s":0.0014819999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015433,"completed_at":"2026-06-14T14:29:06.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [  254.484190] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f331abd8b75cb971cabac0c1","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f331abd8b75cb971cabac0c1: start","job_status":"start","queue_duration_s":0.553552,"scheduling_latency_s":0.55357}1585vm-test-run-nixbot-gitlab> gitlab # [  254.495557] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f331abd8b75cb971cabac0c1","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f331abd8b75cb971cabac0c1: done: 0.015469 sec","job_status":"done","queue_duration_s":0.553552,"scheduling_latency_s":0.55357,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015469,"completed_at":"2026-06-14T14:29:06.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [  254.531960] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc93d5037c18b6853f242fb0","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc93d5037c18b6853f242fb0: start","job_status":"start","queue_duration_s":0.571481,"scheduling_latency_s":0.571501}1587vm-test-run-nixbot-gitlab> gitlab # [  254.543524] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc93d5037c18b6853f242fb0","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc93d5037c18b6853f242fb0: done: 0.016026 sec","job_status":"done","queue_duration_s":0.571481,"scheduling_latency_s":0.571501,"redis_calls":13,"redis_duration_s":0.001645,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016026,"completed_at":"2026-06-14T14:29:06.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [  254.580614] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0c8f41f39965a5be16c12c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0c8f41f39965a5be16c12c: start","job_status":"start","queue_duration_s":0.588646,"scheduling_latency_s":0.588664}1589vm-test-run-nixbot-gitlab> gitlab # [  254.592452] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0c8f41f39965a5be16c12c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0c8f41f39965a5be16c12c: done: 0.015314 sec","job_status":"done","queue_duration_s":0.588646,"scheduling_latency_s":0.588664,"redis_calls":13,"redis_duration_s":0.0015179999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-06-14T14:29:06.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [  254.629546] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"437842e4ee21348bdfddf650","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-437842e4ee21348bdfddf650: start","job_status":"start","queue_duration_s":0.605029,"scheduling_latency_s":0.605047}1591vm-test-run-nixbot-gitlab> gitlab # [  254.641662] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"437842e4ee21348bdfddf650","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-437842e4ee21348bdfddf650: done: 0.014908 sec","job_status":"done","queue_duration_s":0.605029,"scheduling_latency_s":0.605047,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014908,"completed_at":"2026-06-14T14:29:06.613Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [  254.678970] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64da6a39242c0f1efecb977a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64da6a39242c0f1efecb977a: start","job_status":"start","queue_duration_s":0.621024,"scheduling_latency_s":0.62104}1593vm-test-run-nixbot-gitlab> gitlab # [  254.691986] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64da6a39242c0f1efecb977a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64da6a39242c0f1efecb977a: done: 0.014905 sec","job_status":"done","queue_duration_s":0.621024,"scheduling_latency_s":0.62104,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014905,"completed_at":"2026-06-14T14:29:06.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [  254.729119] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15bc1013883ec75f3f3a531c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15bc1013883ec75f3f3a531c: start","job_status":"start","queue_duration_s":0.64079,"scheduling_latency_s":0.640808}1595vm-test-run-nixbot-gitlab> gitlab # [  254.740604] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15bc1013883ec75f3f3a531c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15bc1013883ec75f3f3a531c: done: 0.015356 sec","job_status":"done","queue_duration_s":0.64079,"scheduling_latency_s":0.640808,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015356,"completed_at":"2026-06-14T14:29:06.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [  254.777747] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30175c255afde5876fa3af48","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30175c255afde5876fa3af48: start","job_status":"start","queue_duration_s":0.657201,"scheduling_latency_s":0.657218}1597vm-test-run-nixbot-gitlab> gitlab # [  254.789645] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30175c255afde5876fa3af48","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30175c255afde5876fa3af48: done: 0.015452 sec","job_status":"done","queue_duration_s":0.657201,"scheduling_latency_s":0.657218,"redis_calls":13,"redis_duration_s":0.0015630000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015452,"completed_at":"2026-06-14T14:29:06.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [  254.827138] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b798a150e2e6e075bdc11cb","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b798a150e2e6e075bdc11cb: start","job_status":"start","queue_duration_s":0.673728,"scheduling_latency_s":0.673746}1599vm-test-run-nixbot-gitlab> gitlab # [  254.839040] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b798a150e2e6e075bdc11cb","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b798a150e2e6e075bdc11cb: done: 0.015351 sec","job_status":"done","queue_duration_s":0.673728,"scheduling_latency_s":0.673746,"redis_calls":13,"redis_duration_s":0.0016619999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015351,"completed_at":"2026-06-14T14:29:06.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [  254.876503] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ad289d5da6f996c75b7d12b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ad289d5da6f996c75b7d12b: start","job_status":"start","queue_duration_s":0.690142,"scheduling_latency_s":0.690159}1601vm-test-run-nixbot-gitlab> gitlab # [  254.888267] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ad289d5da6f996c75b7d12b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ad289d5da6f996c75b7d12b: done: 0.015652 sec","job_status":"done","queue_duration_s":0.690142,"scheduling_latency_s":0.690159,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015652,"completed_at":"2026-06-14T14:29:06.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [  254.926584] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b295ee97cfb04803be9390a8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b295ee97cfb04803be9390a8: start","job_status":"start","queue_duration_s":0.706867,"scheduling_latency_s":0.706884}1603vm-test-run-nixbot-gitlab> gitlab # [  254.938611] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b295ee97cfb04803be9390a8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b295ee97cfb04803be9390a8: done: 0.015035 sec","job_status":"done","queue_duration_s":0.706867,"scheduling_latency_s":0.706884,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015035,"completed_at":"2026-06-14T14:29:06.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [  254.975794] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"789faf6ce697b0a6ca9329c8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-789faf6ce697b0a6ca9329c8: start","job_status":"start","queue_duration_s":0.722967,"scheduling_latency_s":0.722985}1605vm-test-run-nixbot-gitlab> gitlab # [  254.987827] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"789faf6ce697b0a6ca9329c8","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-789faf6ce697b0a6ca9329c8: done: 0.015376 sec","job_status":"done","queue_duration_s":0.722967,"scheduling_latency_s":0.722985,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015376,"completed_at":"2026-06-14T14:29:06.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606vm-test-run-nixbot-gitlab> gitlab # [  255.024957] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042aa881c434d40b4868af2b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042aa881c434d40b4868af2b: start","job_status":"start","queue_duration_s":0.739525,"scheduling_latency_s":0.739543}1607vm-test-run-nixbot-gitlab> gitlab # [  255.036651] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042aa881c434d40b4868af2b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042aa881c434d40b4868af2b: done: 0.014849 sec","job_status":"done","queue_duration_s":0.739525,"scheduling_latency_s":0.739543,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014849,"completed_at":"2026-06-14T14:29:06.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608vm-test-run-nixbot-gitlab> gitlab # [  255.073898] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabd5494bc45cc5ba2472061","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabd5494bc45cc5ba2472061: start","job_status":"start","queue_duration_s":0.756225,"scheduling_latency_s":0.756245}1609vm-test-run-nixbot-gitlab> gitlab # [  255.085673] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabd5494bc45cc5ba2472061","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabd5494bc45cc5ba2472061: done: 0.014798 sec","job_status":"done","queue_duration_s":0.756225,"scheduling_latency_s":0.756245,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014798,"completed_at":"2026-06-14T14:29:06.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [  255.123849] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b07803f63d5b51720b15eb67","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b07803f63d5b51720b15eb67: start","job_status":"start","queue_duration_s":0.772112,"scheduling_latency_s":0.772129}1611vm-test-run-nixbot-gitlab> gitlab # [  255.135974] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b07803f63d5b51720b15eb67","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b07803f63d5b51720b15eb67: done: 0.015073 sec","job_status":"done","queue_duration_s":0.772112,"scheduling_latency_s":0.772129,"redis_calls":13,"redis_duration_s":0.0015210000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015073,"completed_at":"2026-06-14T14:29:06.780Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [  255.173628] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"026c1fca4ab7bf57f70f372c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-026c1fca4ab7bf57f70f372c: start","job_status":"start","queue_duration_s":0.788261,"scheduling_latency_s":0.788278}1613vm-test-run-nixbot-gitlab> gitlab # [  255.185406] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"026c1fca4ab7bf57f70f372c","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-026c1fca4ab7bf57f70f372c: done: 0.016877 sec","job_status":"done","queue_duration_s":0.788261,"scheduling_latency_s":0.788278,"redis_calls":13,"redis_duration_s":0.004137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003594,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016877,"completed_at":"2026-06-14T14:29:06.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614vm-test-run-nixbot-gitlab> gitlab # [  255.222511] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b4ef3c8953d1feafaa405a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b4ef3c8953d1feafaa405a: start","job_status":"start","queue_duration_s":0.806242,"scheduling_latency_s":0.806259}1615vm-test-run-nixbot-gitlab> gitlab # [  255.234419] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b4ef3c8953d1feafaa405a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b4ef3c8953d1feafaa405a: done: 0.015016 sec","job_status":"done","queue_duration_s":0.806242,"scheduling_latency_s":0.806259,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015016,"completed_at":"2026-06-14T14:29:06.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616vm-test-run-nixbot-gitlab> gitlab # [  255.272025] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5acc333e49e14af5a316e925","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5acc333e49e14af5a316e925: start","job_status":"start","queue_duration_s":0.823195,"scheduling_latency_s":0.823213}1617vm-test-run-nixbot-gitlab> gitlab # [  255.283597] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5acc333e49e14af5a316e925","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5acc333e49e14af5a316e925: done: 0.014751 sec","job_status":"done","queue_duration_s":0.823195,"scheduling_latency_s":0.823213,"redis_calls":13,"redis_duration_s":0.00217,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014751,"completed_at":"2026-06-14T14:29:06.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618vm-test-run-nixbot-gitlab> gitlab # [  255.321358] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21c36ac5e16da6bcd6f3f85","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21c36ac5e16da6bcd6f3f85: start","job_status":"start","queue_duration_s":0.838973,"scheduling_latency_s":0.83899}1619vm-test-run-nixbot-gitlab> gitlab # [  255.334097] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21c36ac5e16da6bcd6f3f85","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21c36ac5e16da6bcd6f3f85: done: 0.014786 sec","job_status":"done","queue_duration_s":0.838973,"scheduling_latency_s":0.83899,"redis_calls":13,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014786,"completed_at":"2026-06-14T14:29:06.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [  255.371055] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db87fad6f6f7d65cc0cb9138","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db87fad6f6f7d65cc0cb9138: start","job_status":"start","queue_duration_s":0.854806,"scheduling_latency_s":0.854823}1621vm-test-run-nixbot-gitlab> gitlab # [  255.382884] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db87fad6f6f7d65cc0cb9138","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db87fad6f6f7d65cc0cb9138: done: 0.014779 sec","job_status":"done","queue_duration_s":0.854806,"scheduling_latency_s":0.854823,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014779,"completed_at":"2026-06-14T14:29:06.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [  255.419721] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1bab5b4f02ed3bcc1c24c40","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1bab5b4f02ed3bcc1c24c40: start","job_status":"start","queue_duration_s":0.8713,"scheduling_latency_s":0.871326}1623vm-test-run-nixbot-gitlab> gitlab # [  255.431605] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1bab5b4f02ed3bcc1c24c40","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1bab5b4f02ed3bcc1c24c40: done: 0.015796 sec","job_status":"done","queue_duration_s":0.8713,"scheduling_latency_s":0.871326,"redis_calls":13,"redis_duration_s":0.0017400000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015796,"completed_at":"2026-06-14T14:29:06.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [  255.468352] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a69ebf063727d580b7470f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a69ebf063727d580b7470f: start","job_status":"start","queue_duration_s":0.887632,"scheduling_latency_s":0.88765}1625vm-test-run-nixbot-gitlab> gitlab # [  255.479907] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a69ebf063727d580b7470f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a69ebf063727d580b7470f: done: 0.014798 sec","job_status":"done","queue_duration_s":0.887632,"scheduling_latency_s":0.88765,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014798,"completed_at":"2026-06-14T14:29:06.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626vm-test-run-nixbot-gitlab> gitlab # [  255.516839] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccb9340164efe0e34fd4734b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccb9340164efe0e34fd4734b: start","job_status":"start","queue_duration_s":0.903495,"scheduling_latency_s":0.903512}1627vm-test-run-nixbot-gitlab> gitlab # [  255.528438] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccb9340164efe0e34fd4734b","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccb9340164efe0e34fd4734b: done: 0.015561 sec","job_status":"done","queue_duration_s":0.903495,"scheduling_latency_s":0.903512,"redis_calls":13,"redis_duration_s":0.0014399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015561,"completed_at":"2026-06-14T14:29:06.912Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628vm-test-run-nixbot-gitlab> gitlab # [  255.566428] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6f717cbd898097e9b7c6769","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6f717cbd898097e9b7c6769: start","job_status":"start","queue_duration_s":0.920178,"scheduling_latency_s":0.920195}1629vm-test-run-nixbot-gitlab> gitlab # [  255.578062] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6f717cbd898097e9b7c6769","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6f717cbd898097e9b7c6769: done: 0.015577 sec","job_status":"done","queue_duration_s":0.920178,"scheduling_latency_s":0.920195,"redis_calls":14,"redis_duration_s":0.0016350000000000002,"redis_read_bytes":138,"redis_write_bytes":1221,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000115,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015577,"completed_at":"2026-06-14T14:29:06.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [  255.617020] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2b67d95ee6e62c2350f7419","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2b67d95ee6e62c2350f7419: start","job_status":"start","queue_duration_s":0.936821,"scheduling_latency_s":0.936838}1631vm-test-run-nixbot-gitlab> gitlab # [  255.628704] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2b67d95ee6e62c2350f7419","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2b67d95ee6e62c2350f7419: done: 0.01475 sec","job_status":"done","queue_duration_s":0.936821,"scheduling_latency_s":0.936838,"redis_calls":13,"redis_duration_s":0.0014620000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01475,"completed_at":"2026-06-14T14:29:06.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [  255.666284] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2a79ac84c2493f8d5fda5d6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2a79ac84c2493f8d5fda5d6: start","job_status":"start","queue_duration_s":0.95264,"scheduling_latency_s":0.952658}1633vm-test-run-nixbot-gitlab> gitlab # [  255.678170] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2a79ac84c2493f8d5fda5d6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2a79ac84c2493f8d5fda5d6: done: 0.015866 sec","job_status":"done","queue_duration_s":0.95264,"scheduling_latency_s":0.952658,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015866,"completed_at":"2026-06-14T14:29:06.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634vm-test-run-nixbot-gitlab> gitlab # [  255.715479] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a843bd5f5744871402fe75a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a843bd5f5744871402fe75a: start","job_status":"start","queue_duration_s":0.969632,"scheduling_latency_s":0.969649}1635vm-test-run-nixbot-gitlab> gitlab # [  255.727436] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a843bd5f5744871402fe75a","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a843bd5f5744871402fe75a: done: 0.014704 sec","job_status":"done","queue_duration_s":0.969632,"scheduling_latency_s":0.969649,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014704,"completed_at":"2026-06-14T14:29:06.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [  255.765437] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"171f5dd2ef540ac8dde6d91d","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-171f5dd2ef540ac8dde6d91d: start","job_status":"start","queue_duration_s":0.985788,"scheduling_latency_s":0.985806}1637vm-test-run-nixbot-gitlab> gitlab # [  255.777213] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"171f5dd2ef540ac8dde6d91d","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-171f5dd2ef540ac8dde6d91d: done: 0.014732 sec","job_status":"done","queue_duration_s":0.985788,"scheduling_latency_s":0.985806,"redis_calls":13,"redis_duration_s":0.002597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014732,"completed_at":"2026-06-14T14:29:06.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [  255.814473] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:06.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25dd7aec22c6513cbb7ebf89","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25dd7aec22c6513cbb7ebf89: start","job_status":"start","queue_duration_s":1.001623,"scheduling_latency_s":1.001638}1639vm-test-run-nixbot-gitlab> gitlab # [  255.826411] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25dd7aec22c6513cbb7ebf89","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25dd7aec22c6513cbb7ebf89: done: 0.015025 sec","job_status":"done","queue_duration_s":1.001623,"scheduling_latency_s":1.001638,"redis_calls":13,"redis_duration_s":0.001997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015025,"completed_at":"2026-06-14T14:29:07.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [  255.863631] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73420b9b706914da7325eaca","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73420b9b706914da7325eaca: start","job_status":"start","queue_duration_s":1.018552,"scheduling_latency_s":1.018569}1641vm-test-run-nixbot-gitlab> gitlab # [  255.875665] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73420b9b706914da7325eaca","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73420b9b706914da7325eaca: done: 0.014765 sec","job_status":"done","queue_duration_s":1.018552,"scheduling_latency_s":1.018569,"redis_calls":13,"redis_duration_s":0.002288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001504,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-06-14T14:29:07.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [  255.912946] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1166b9af7401243d262d9051","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1166b9af7401243d262d9051: start","job_status":"start","queue_duration_s":1.034545,"scheduling_latency_s":1.034562}1643vm-test-run-nixbot-gitlab> gitlab # [  255.924615] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1166b9af7401243d262d9051","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1166b9af7401243d262d9051: done: 0.015538 sec","job_status":"done","queue_duration_s":1.034545,"scheduling_latency_s":1.034562,"redis_calls":13,"redis_duration_s":0.002496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015538,"completed_at":"2026-06-14T14:29:07.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [  255.961456] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6683608020340cc129836719","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6683608020340cc129836719: start","job_status":"start","queue_duration_s":1.052062,"scheduling_latency_s":1.052081}1645vm-test-run-nixbot-gitlab> gitlab # [  255.974092] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6683608020340cc129836719","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6683608020340cc129836719: done: 0.014709 sec","job_status":"done","queue_duration_s":1.052062,"scheduling_latency_s":1.052081,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014709,"completed_at":"2026-06-14T14:29:07.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [  256.011439] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcd9d35b602850cf7c08153","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcd9d35b602850cf7c08153: start","job_status":"start","queue_duration_s":1.068232,"scheduling_latency_s":1.068251}1647vm-test-run-nixbot-gitlab> gitlab # [  256.023642] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcd9d35b602850cf7c08153","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcd9d35b602850cf7c08153: done: 0.015294 sec","job_status":"done","queue_duration_s":1.068232,"scheduling_latency_s":1.068251,"redis_calls":13,"redis_duration_s":0.001528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015294,"completed_at":"2026-06-14T14:29:07.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [  256.060920] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3537ff8e80d5dd268d3adb8f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3537ff8e80d5dd268d3adb8f: start","job_status":"start","queue_duration_s":1.084675,"scheduling_latency_s":1.084693}1649vm-test-run-nixbot-gitlab> gitlab # [  256.072583] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3537ff8e80d5dd268d3adb8f","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3537ff8e80d5dd268d3adb8f: done: 0.015974 sec","job_status":"done","queue_duration_s":1.084675,"scheduling_latency_s":1.084693,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015974,"completed_at":"2026-06-14T14:29:07.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [  256.109371] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd9c3ffbcc246eddf5c672b6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd9c3ffbcc246eddf5c672b6: start","job_status":"start","queue_duration_s":1.101738,"scheduling_latency_s":1.101758}1651vm-test-run-nixbot-gitlab> gitlab # [  256.121088] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd9c3ffbcc246eddf5c672b6","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd9c3ffbcc246eddf5c672b6: done: 0.014829 sec","job_status":"done","queue_duration_s":1.101738,"scheduling_latency_s":1.101758,"redis_calls":13,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014829,"completed_at":"2026-06-14T14:29:07.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [  256.158047] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9366f01294c54ae5ee195279","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9366f01294c54ae5ee195279: start","job_status":"start","queue_duration_s":1.11762,"scheduling_latency_s":1.117637}1653vm-test-run-nixbot-gitlab> gitlab # [  256.169753] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9366f01294c54ae5ee195279","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9366f01294c54ae5ee195279: done: 0.015049 sec","job_status":"done","queue_duration_s":1.11762,"scheduling_latency_s":1.117637,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015049,"completed_at":"2026-06-14T14:29:07.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [  256.207728] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5b64dda1e69f5e3a8574c2","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5b64dda1e69f5e3a8574c2: start","job_status":"start","queue_duration_s":1.133857,"scheduling_latency_s":1.133874}1655vm-test-run-nixbot-gitlab> gitlab # [  256.219603] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:07.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5b64dda1e69f5e3a8574c2","created_at":"2026-06-14T14:29:05.914Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-107621c7a6b14b1a","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b97b3cba8244ace8844a37b312430d48","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:05.993Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5b64dda1e69f5e3a8574c2: done: 0.014517 sec","job_status":"done","queue_duration_s":1.133857,"scheduling_latency_s":1.133874,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014517,"completed_at":"2026-06-14T14:29:07.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [  257.156681] setup-gitlab-start[3303]: W, [2026-06-14T14:29:10.056144 #3303]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1657vm-test-run-nixbot-gitlab> gitlab # [  296.451598] runuser[3302]: pam_unix(runuser:session): session closed for user gitlab1658vm-test-run-nixbot-gitlab> gitlab # [  296.456747] setup-gitlab-start[3637]: ++ jq -r .token1659vm-test-run-nixbot-gitlab> gitlab # [  296.459952] setup-gitlab-start[3638]: +++ date -d '+7 days' +%F1660vm-test-run-nixbot-gitlab> gitlab # [  296.463216] setup-gitlab-start[3636]: ++ curl -fs -X POST -H 'Authorization: Bearer e502fa590c4fa78e4abaa79e96310ac3232c33a4e656e2e456bcf5f7c5f36197' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-06-21"}' http://gitlab/api/v4/users/1/personal_access_tokens1661vm-test-run-nixbot-gitlab> gitlab # [  296.940817] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:49.839Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"02a40f62fca91746acc4b5eb","created_at":"2026-06-14T14:29:49.781Z","trace_propagation_headers":{"sentry-trace":"6b4b6d5b96614e6ba45118eac981a80f-29be7cae8d7e4fb1","baggage":"sentry-trace_id=6b4b6d5b96614e6ba45118eac981a80f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KW6DRQ1PJRTXC59PAV41","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c7da652b7ac161a1661d9b00df6cbce7bcb097519b8f90b8ce3e2c728af3108","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:49.834Z","job_size_bytes":451,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-02a40f62fca91746acc4b5eb: start","job_status":"start","queue_duration_s":0.004388,"scheduling_latency_s":0.004451}1662vm-test-run-nixbot-gitlab> gitlab # [  297.000097] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:49 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 5101663vm-test-run-nixbot-gitlab> gitlab # [  297.007992] setup-gitlab-start[3256]: + TOKEN=glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p1664vm-test-run-nixbot-gitlab> gitlab # [  297.011365] setup-gitlab-start[3256]: + test -n glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p1665vm-test-run-nixbot-gitlab> gitlab # [  297.014896] setup-gitlab-start[3256]: + test glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p '!=' null1666vm-test-run-nixbot-gitlab> gitlab # [  297.018572] setup-gitlab-start[3256]: + mkdir -p /var/lib/secrets1667vm-test-run-nixbot-gitlab> gitlab # [  297.021047] setup-gitlab-start[3256]: + echo glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p1668vm-test-run-nixbot-gitlab> gitlab # [  297.024354] setup-gitlab-start[3256]: + chmod 644 /var/lib/secrets/gitlab-token1669vm-test-run-nixbot-gitlab> gitlab # [  297.027102] setup-gitlab-start[3256]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1670vm-test-run-nixbot-gitlab> gitlab # [  297.223137] sidekiq-cluster[2418]: {"severity":"WARN","time":"2026-06-14T14:29:50.122Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"02a40f62fca91746acc4b5eb","created_at":"2026-06-14T14:29:49.781Z","trace_propagation_headers":{"sentry-trace":"6b4b6d5b96614e6ba45118eac981a80f-29be7cae8d7e4fb1","baggage":"sentry-trace_id=6b4b6d5b96614e6ba45118eac981a80f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KW6DRQ1PJRTXC59PAV41","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c7da652b7ac161a1661d9b00df6cbce7bcb097519b8f90b8ce3e2c728af3108","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:49.834Z","job_size_bytes":451,"pid":2418,"sidekiq_tid":"lyly","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-02a40f62fca91746acc4b5eb: fail: 0.282944 sec","job_status":"fail","queue_duration_s":0.004388,"scheduling_latency_s":0.004451,"redis_calls":17,"redis_duration_s":0.016724,"redis_read_bytes":651,"redis_write_bytes":1842,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.014,"redis_feature_flag_read_bytes":644,"redis_feature_flag_write_bytes":958,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061674,"duration_s":0.282944,"completed_at":"2026-06-14T14:29:50.121Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.063844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1671vm-test-run-nixbot-gitlab> gitlab # [  298.326245] gitaly[1356]: time="2026-06-14T14:29:51.225Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.225" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672vm-test-run-nixbot-gitlab> gitlab # [  298.326721] gitaly[1356]: time="2026-06-14T14:29:51.226Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.225" grpc.time_ms=0.329 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1673vm-test-run-nixbot-gitlab> gitlab # [  298.822597] gitaly[1356]: time="2026-06-14T14:29:51.723Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.722" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1674vm-test-run-nixbot-gitlab> gitlab # [  298.850598] gitaly[1356]: time="2026-06-14T14:29:51.723Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.722" grpc.time_ms=0.523 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675vm-test-run-nixbot-gitlab> gitlab # [  298.883034] gitaly[1356]: time="2026-06-14T14:29:51.764Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 diskcache=ef5eeaa6-7206-40c2-ab91-0eb09df60268 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T14:30:01.014" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.735" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1676vm-test-run-nixbot-gitlab> gitlab # [  298.905839] gitaly[1356]: time="2026-06-14T14:29:51.764Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T14:30:01.014" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.735" grpc.time_ms=28.986 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1677vm-test-run-nixbot-gitlab> gitlab # [  298.927775] gitaly[1356]: time="2026-06-14T14:29:51.785Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.064" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.784" pid=1356 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1678vm-test-run-nixbot-gitlab> gitlab # [  298.942955] gitaly[1356]: time="2026-06-14T14:29:51.786Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.064" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.784" grpc.time_ms=1.367 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679vm-test-run-nixbot-gitlab> gitlab # [  298.958411] gitaly[1356]: time="2026-06-14T14:29:51.815Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:01.077" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.798" grpc.time_ms=16.457 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680vm-test-run-nixbot-gitlab> gitlab # [  299.012982] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:51.911Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"9f0b4295cd8bbed94d51ed82","created_at":"2026-06-14T14:29:51.835Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:51.902Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-9f0b4295cd8bbed94d51ed82: start","job_status":"start","queue_duration_s":0.00959,"scheduling_latency_s":0.009647}1681vm-test-run-nixbot-gitlab> gitlab # [  299.068574] gitaly[1356]: time="2026-06-14T14:29:51.969Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.248" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.968" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [  299.078670] gitaly[1356]: time="2026-06-14T14:29:51.969Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:01.248" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.968" grpc.time_ms=0.62 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1683vm-test-run-nixbot-gitlab> gitlab # [  299.108592] gitaly[1356]: time="2026-06-14T14:29:52.009Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 diskcache=1dc28fdc-0d6e-4dc1-bca5-668358594bdf grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T14:30:01.260" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.981" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1684vm-test-run-nixbot-gitlab> gitlab # [  299.118837] gitaly[1356]: time="2026-06-14T14:29:52.009Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T14:30:01.260" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:51.981" grpc.time_ms=28.224 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685vm-test-run-nixbot-gitlab> gitlab # [  299.134107] gitaly[1356]: time="2026-06-14T14:29:52.034Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:02.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:52.033" pid=1356 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1686vm-test-run-nixbot-gitlab> gitlab # [  299.144467] gitaly[1356]: time="2026-06-14T14:29:52.035Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:02.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:52.033" grpc.time_ms=1.557 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687vm-test-run-nixbot-gitlab> gitlab # [  299.155023] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:52.048Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"9f0b4295cd8bbed94d51ed82","created_at":"2026-06-14T14:29:51.835Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:51.902Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-9f0b4295cd8bbed94d51ed82: done: 0.136777 sec","job_status":"done","queue_duration_s":0.00959,"scheduling_latency_s":0.009647,"redis_calls":15,"redis_duration_s":0.00961,"redis_read_bytes":140,"redis_write_bytes":1876,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002256,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":711,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005623,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001731,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07505874,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.136777,"completed_at":"2026-06-14T14:29:52.048Z","load_balancing_strategy":"primary","db_duration_s":0.016838,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1688vm-test-run-nixbot-gitlab> gitlab # [  299.197550] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:52.052Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"15742c2580b7493eec92a912","created_at":"2026-06-14T14:29:51.937Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:51.953Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-15742c2580b7493eec92a912: start","job_status":"start","queue_duration_s":0.098743,"scheduling_latency_s":0.098801}1689vm-test-run-nixbot-gitlab> gitlab # [  299.211593] gitaly[1356]: time="2026-06-14T14:29:52.058Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:02.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:52.044" grpc.time_ms=13.694 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690vm-test-run-nixbot-gitlab> gitlab # [  299.238983] gitaly[1356]: time="2026-06-14T14:29:52.139Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:02.108" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:52.139" grpc.time_ms=0.208 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1691vm-test-run-nixbot-gitlab> gitlab # [  299.304539] gitaly[1356]: time="2026-06-14T14:29:52.205Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T14:30:02.173" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:52.204" grpc.time_ms=0.528 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692vm-test-run-nixbot-gitlab> gitlab # [  299.317245] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:52.217Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"15742c2580b7493eec92a912","created_at":"2026-06-14T14:29:51.937Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:51.953Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-15742c2580b7493eec92a912: done: 0.164873 sec","job_status":"done","queue_duration_s":0.098743,"scheduling_latency_s":0.098801,"redis_calls":14,"redis_duration_s":0.006889999999999999,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001488,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003286,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1043383,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.164873,"completed_at":"2026-06-14T14:29:52.217Z","load_balancing_strategy":"primary","db_duration_s":0.035485,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1693vm-test-run-nixbot-gitlab> gitlab # [  301.120514] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.019Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"41fe83bdcda55f670f4f4449","created_at":"2026-06-14T14:29:54.005Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.017Z","job_size_bytes":5,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-41fe83bdcda55f670f4f4449: start","job_status":"start","queue_duration_s":0.002172,"scheduling_latency_s":0.002229}1694vm-test-run-nixbot-gitlab> gitlab # [  301.239542] gitaly[1356]: time="2026-06-14T14:29:54.139Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KV38KWQGS82RTKWJ64HSRJB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:04.095" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:54.125" grpc.time_ms=13.561 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695vm-test-run-nixbot-gitlab> gitlab # [  301.340800] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.240Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"41fe83bdcda55f670f4f4449","created_at":"2026-06-14T14:29:54.005Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.017Z","job_size_bytes":5,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-41fe83bdcda55f670f4f4449: done: 0.221472 sec","job_status":"done","queue_duration_s":0.002172,"scheduling_latency_s":0.002229,"redis_calls":12,"redis_duration_s":0.0047740000000000005,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00103,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002904,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00084,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.179737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078374,"duration_s":0.221472,"completed_at":"2026-06-14T14:29:54.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032683,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1696vm-test-run-nixbot-gitlab> gitlab # [  301.438267] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.243Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"0de429d1d2ff478d7fd855c8","created_at":"2026-06-14T14:29:54.073Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.080Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-0de429d1d2ff478d7fd855c8: start","job_status":"start","queue_duration_s":0.162608,"scheduling_latency_s":0.162642}1697vm-test-run-nixbot-gitlab> gitlab # [  301.731313] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.630Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"0de429d1d2ff478d7fd855c8","created_at":"2026-06-14T14:29:54.073Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.080Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-0de429d1d2ff478d7fd855c8: done: 0.387207 sec","job_status":"done","queue_duration_s":0.162608,"scheduling_latency_s":0.162642,"redis_calls":11,"redis_duration_s":0.003423,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000881,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.333,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.333,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.298127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049248,"duration_s":0.387207,"completed_at":"2026-06-14T14:29:54.630Z","load_balancing_strategy":"primary","db_duration_s":0.095404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1698vm-test-run-nixbot-gitlab> gitlab # [  301.817921] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.633Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8d5fcb20ec206f44789e9944","created_at":"2026-06-14T14:29:54.086Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.099Z","job_size_bytes":89,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8d5fcb20ec206f44789e9944: start","job_status":"start","queue_duration_s":0.533838,"scheduling_latency_s":0.533864}1699vm-test-run-nixbot-gitlab> gitlab # [  301.841061] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:54.731Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8d5fcb20ec206f44789e9944","created_at":"2026-06-14T14:29:54.086Z","trace_propagation_headers":{"sentry-trace":"570c2e40d7f04aa093d1f25e41b29fdb-8be1ae3e37684e8b","baggage":"sentry-trace_id=570c2e40d7f04aa093d1f25e41b29fdb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38KWQGS82RTKWJ64HSRJB6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:54.099Z","job_size_bytes":89,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8d5fcb20ec206f44789e9944: done: 0.098331 sec","job_status":"done","queue_duration_s":0.533838,"scheduling_latency_s":0.533864,"redis_calls":11,"redis_duration_s":0.003253,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000842,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064328,"duration_s":0.098331,"completed_at":"2026-06-14T14:29:54.731Z","load_balancing_strategy":"primary","db_duration_s":0.038492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1700vm-test-run-nixbot-gitlab> gitlab # [  301.902542] setup-gitlab-start[3256]: + mkdir -p /tmp/test-flake1701vm-test-run-nixbot-gitlab> gitlab # [  301.903616] setup-gitlab-start[3256]: + cd /tmp/test-flake1702vm-test-run-nixbot-gitlab> gitlab # [  301.904546] setup-gitlab-start[3256]: + git init -b master1703vm-test-run-nixbot-gitlab> gitlab # [  301.905806] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:54 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 47961704vm-test-run-nixbot-gitlab> gitlab # [  301.908092] setup-gitlab-start[3691]: Initialized empty Git repository in /tmp/test-flake/.git/1705vm-test-run-nixbot-gitlab> gitlab # [  301.909440] setup-gitlab-start[3256]: + git config user.name test1706vm-test-run-nixbot-gitlab> gitlab # [  301.910394] setup-gitlab-start[3256]: + git config user.email test@example.com1707vm-test-run-nixbot-gitlab> gitlab # [  301.911508] setup-gitlab-start[3256]: + cat1708vm-test-run-nixbot-gitlab> gitlab # [  301.912222] setup-gitlab-start[3256]: + git add flake.nix1709vm-test-run-nixbot-gitlab> gitlab # [  301.913065] setup-gitlab-start[3256]: + git commit -m 'initial commit'1710vm-test-run-nixbot-gitlab> gitlab # [  301.914243] setup-gitlab-start[3696]: [master (root-commit) 5e087cd] initial commit1711vm-test-run-nixbot-gitlab> gitlab # [  301.915420] setup-gitlab-start[3696]:  1 file changed, 10 insertions(+)1712vm-test-run-nixbot-gitlab> gitlab # [  301.916460] setup-gitlab-start[3696]:  create mode 100644 flake.nix1713vm-test-run-nixbot-gitlab> gitlab # [  301.917457] setup-gitlab-start[3256]: + git remote add origin http://oauth2:glpat-I88pmrI6pVD01g7Je8ra5W86MQp1OjEH.01.0w0l3i15p@gitlab/root/test-flake.git1714vm-test-run-nixbot-gitlab> gitlab # [  301.919440] setup-gitlab-start[3256]: + git push -u origin master1715vm-test-run-nixbot-gitlab> gitlab # [  302.110665] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 1591716vm-test-run-nixbot-gitlab> gitlab # [  302.310931] gitaly[1356]: time="2026-06-14T14:29:55.210Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1P4CVB7DPTS8GARCF3R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:29:55.192" grpc.time_ms=17.466 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1717vm-test-run-nixbot-gitlab> gitlab # [  302.335132] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 2001718vm-test-run-nixbot-gitlab> gitlab # [  302.648492] gitaly[1356]: time="2026-06-14T14:29:55.549Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:05.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:55.533" grpc.time_ms=15.779 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1719vm-test-run-nixbot-gitlab> gitlab # [  302.723394] gitaly[1356]: time="2026-06-14T14:29:55.624Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-14T14:30:25.006" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-14T14:29:55.606" grpc.time_ms=17.166 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720vm-test-run-nixbot-gitlab> gitlab # [  302.765091] gitaly[1356]: time="2026-06-14T14:29:55.665Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV38M1WT4F6B1829DD7HNTNJ duration_ms=227ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/allowed"1721vm-test-run-nixbot-gitlab> gitlab # [  302.824516] gitaly[1356]: time="2026-06-14T14:29:55.725Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV38M1WT4F6B1829DD7HNTNJ duration_ms=55ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/pre_receive"1722vm-test-run-nixbot-gitlab> gitlab # [  302.833648] gitaly[1356]: time="2026-06-14T14:29:55.725Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:55.437" grpc.time_ms=288.409 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1723vm-test-run-nixbot-gitlab> gitlab # [  302.877908] gitaly[1356]: time="2026-06-14T14:29:55.778Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:55.777" grpc.time_ms=1.275 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1724vm-test-run-nixbot-gitlab> gitlab # [  302.957010] gitaly[1356]: time="2026-06-14T14:29:55.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:55.856" grpc.time_ms=0.964 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1725vm-test-run-nixbot-gitlab> gitlab # [  303.008628] gitaly[1356]: time="2026-06-14T14:29:55.909Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:55.908" grpc.time_ms=0.821 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1726vm-test-run-nixbot-gitlab> gitlab # [  303.115330] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:56.014Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5e087cdad9dd54e78315af82c4122a879a9fec06 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"27faf40731818addcce67a1a","created_at":"2026-06-14T14:29:56.002Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-959dd35ae3144af5","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.010Z","job_size_bytes":185,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-27faf40731818addcce67a1a: start","job_status":"start","queue_duration_s":0.002893,"scheduling_latency_s":0.00297}1727vm-test-run-nixbot-gitlab> gitlab # [  303.161233] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:56.058Z","message":"changes: \"0000000000000000000000000000000000000000 5e087cdad9dd54e78315af82c4122a879a9fec06 refs/heads/master\\n\""}1728vm-test-run-nixbot-gitlab> gitlab # [  303.178455] gitaly[1356]: time="2026-06-14T14:29:56.079Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=41 command.spawn_token_fork_ms=28 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:06.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.035" grpc.time_ms=42.882 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729vm-test-run-nixbot-gitlab> gitlab # [  303.216981] gitaly[1356]: time="2026-06-14T14:29:56.117Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:06.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.089" grpc.time_ms=27.518 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730vm-test-run-nixbot-gitlab> gitlab # [  303.244994] gitaly[1356]: time="2026-06-14T14:29:56.124Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1436 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-14T14:30:06.065" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.095" grpc.time_ms=27.87 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1731vm-test-run-nixbot-gitlab> gitlab # [  303.268923] gitaly[1356]: time="2026-06-14T14:29:56.142Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV38M1WT4F6B1829DD7HNTNJ duration_ms=187ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/post_receive"1732vm-test-run-nixbot-gitlab> gitlab # [  303.273868] gitaly[1356]: time="2026-06-14T14:29:56.143Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:55.953" grpc.time_ms=189.7 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1733vm-test-run-nixbot-gitlab> gitlab # [  303.288961] gitaly[1356]: time="2026-06-14T14:29:56.152Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ diskcache=ec957fa6-bbb1-41dd-87ce-1d98c0286ba0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:29:55.340" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1734vm-test-run-nixbot-gitlab> gitlab # [  303.301850] systemd[1]: Finished setup-gitlab.service.1735vm-test-run-nixbot-gitlab> gitlab # [  303.303174] gitlab-workhorse[1286]: time="2026-06-14T14:29:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521736vm-test-run-nixbot-gitlab> gitlab # [  303.305418] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:56 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9261737vm-test-run-nixbot-gitlab> gitlab # [  303.307896] setup-gitlab-start[3700]: To http://gitlab/root/test-flake.git1738vm-test-run-nixbot-gitlab> gitlab # [  303.308990] setup-gitlab-start[3700]:  * [new branch]      master -> master1739vm-test-run-nixbot-gitlab> gitlab # [  303.310021] setup-gitlab-start[3700]: branch 'master' set up to track 'origin/master'.1740vm-test-run-nixbot-gitlab> gitlab # [  303.311509] gitaly[1356]: time="2026-06-14T14:29:56.152Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=297 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13020 command.oublock=32 command.real_time_ms=804 command.spawn_token_fork_ms=1 command.system_time_ms=186 command.user_time_ms=111 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:29:55.340" grpc.time_ms=811.745 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1741vm-test-run-nixbot-gitlab> gitlab # [  303.326014] systemd[1]: Starting nixbot CI service...1742vm-test-run-nixbot-gitlab> gitlab # [  303.327674] gitaly[1356]: time="2026-06-14T14:29:56.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:06.075" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.156" grpc.time_ms=9.691 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743vm-test-run-nixbot-gitlab> gitlab # [  303.340776] gitaly[1356]: time="2026-06-14T14:29:56.235Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-14T14:30:06.142" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.222" grpc.time_ms=12.856 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744vm-test-run-nixbot-gitlab> gitlab # [  303.398728] gitaly[1356]: time="2026-06-14T14:29:56.298Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3936 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:56.297" grpc.time_ms=0.521 pid=1356 span.kind=server system=grpc1745vm-test-run-nixbot-gitlab> gitlab # [  303.448679] gitaly[1356]: time="2026-06-14T14:29:56.349Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:29:56.348" grpc.time_ms=0.809 pid=1356 span.kind=server system=grpc1746vm-test-run-nixbot-gitlab> gitlab # [  303.468176] gitaly[1356]: time="2026-06-14T14:29:56.358Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ diskcache=5701d3dc-91e6-4159-aa23-69d9b50f693f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:06.169" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.250" pid=1356 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1747vm-test-run-nixbot-gitlab> gitlab # [  303.485059] gitaly[1356]: time="2026-06-14T14:29:56.358Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=105 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5204 command.oublock=8 command.real_time_ms=103 command.spawn_token_fork_ms=0 command.system_time_ms=70 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:06.169" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.250" grpc.time_ms=108.266 pid=1356 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1748vm-test-run-nixbot-gitlab> gitlab # [  303.504466] gitaly[1356]: time="2026-06-14T14:29:56.385Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T14:30:06.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.385" grpc.time_ms=0.122 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 304.13 seconds)1750vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1751vm-test-run-nixbot-gitlab> gitlab # [  303.520716] gitaly[1356]: time="2026-06-14T14:29:56.399Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=862 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-14T14:30:06.009" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.389" grpc.time_ms=10.312 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752vm-test-run-nixbot-gitlab> gitlab # [  303.595211] gitaly[1356]: time="2026-06-14T14:29:56.495Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-14T14:30:26.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:56.490" grpc.time_ms=5.689 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753vm-test-run-nixbot-gitlab> gitlab # [  303.631842] gitaly[1356]: time="2026-06-14T14:29:56.524Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T14:30:06.143" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:56.524" grpc.time_ms=0.245 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [  303.655232] gitaly[1356]: time="2026-06-14T14:29:56.531Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-14T14:30:06.146" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-14T14:29:56.526" grpc.time_ms=5.082 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755vm-test-run-nixbot-gitlab> gitlab # [  303.678265] gitaly[1356]: time="2026-06-14T14:29:56.554Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:06.166" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.547" grpc.time_ms=7.372 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756vm-test-run-nixbot-gitlab> gitlab # [  303.699735] gitaly[1356]: time="2026-06-14T14:29:56.566Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:06.179" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:56.559" grpc.time_ms=7.423 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757vm-test-run-nixbot-gitlab> gitlab # [  303.722212] gitaly[1356]: time="2026-06-14T14:29:56.595Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=884 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T14:30:26.178" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:56.579" grpc.time_ms=16.294 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [  304.331743] gitaly[1356]: time="2026-06-14T14:29:57.231Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-14T14:30:07.147" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.228" grpc.time_ms=2.875 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759vm-test-run-nixbot-gitlab> gitlab # [  304.397019] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.295Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5e087cdad9dd54e78315af82c4122a879a9fec06 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"27faf40731818addcce67a1a","created_at":"2026-06-14T14:29:56.002Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-959dd35ae3144af5","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.010Z","job_size_bytes":185,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-27faf40731818addcce67a1a: done: 1.282031 sec","job_status":"done","queue_duration_s":0.002893,"scheduling_latency_s":0.00297,"gitaly_calls":13,"gitaly_duration_s":0.256727,"redis_calls":200,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.075947,"redis_read_bytes":941,"redis_write_bytes":19810,"redis_cache_calls":14,"redis_cache_duration_s":0.004642,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.011113,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.002601,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.009886,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.028135,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01957,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.069,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.294,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.319,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.069,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.736833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00119644,"exclusive_lock_hold_duration_s":0.05246774,"duration_s":1.282031,"completed_at":"2026-06-14T14:29:57.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.106924,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1760vm-test-run-nixbot-gitlab> gitlab # [  304.525331] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.298Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"856e4e1442a00a51cc70e09e","created_at":"2026-06-14T14:29:56.173Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.182Z","job_size_bytes":46,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-856e4e1442a00a51cc70e09e: start","job_status":"start","queue_duration_s":1.115523,"scheduling_latency_s":1.115559}1761vm-test-run-nixbot-gitlab> gitlab # [  304.555538] gitaly[1356]: time="2026-06-14T14:29:57.381Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-14T20:29:57.375" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:29:57.376" grpc.time_ms=4.868 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762vm-test-run-nixbot-gitlab> gitlab # [  304.574250] nixbot[3790]: {"ts": "2026-06-14T14:29:57.445Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1763vm-test-run-nixbot-gitlab> gitlab # [  304.577643] nixbot[3790]: {"ts": "2026-06-14T14:29:57.466Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1764vm-test-run-nixbot-gitlab> gitlab # [  304.580671] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.447Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"856e4e1442a00a51cc70e09e","created_at":"2026-06-14T14:29:56.173Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.182Z","job_size_bytes":46,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-856e4e1442a00a51cc70e09e: done: 0.149444 sec","job_status":"done","queue_duration_s":1.115523,"scheduling_latency_s":1.115559,"gitaly_calls":2,"gitaly_duration_s":0.02791,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010015,"redis_read_bytes":68,"redis_write_bytes":4884,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002035,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00069,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2113,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001973,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004284,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004654,"duration_s":0.149444,"completed_at":"2026-06-14T14:29:57.447Z","load_balancing_strategy":"primary","db_duration_s":0.018483,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1765vm-test-run-nixbot-gitlab> gitlab # [  304.646235] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.449Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"027f6985b73bdc039b2f648c","created_at":"2026-06-14T14:29:56.207Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.216Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-027f6985b73bdc039b2f648c: start","job_status":"start","queue_duration_s":1.232845,"scheduling_latency_s":1.232871}1766vm-test-run-nixbot-gitlab> gitlab # [  304.664486] gitaly[1356]: time="2026-06-14T14:29:57.410Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-14T14:30:27.092" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.393" grpc.time_ms=16.669 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767vm-test-run-nixbot-gitlab> gitlab # [  304.677866] gitaly[1356]: time="2026-06-14T14:29:57.508Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ diskcache=3c40c9a9-c1b1-4e3e-8a05-861d094f1588 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T20:29:57.480" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.481" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768vm-test-run-nixbot-gitlab> gitlab # [  304.688600] nixbot[3790]: {"ts": "2026-06-14T14:29:57.480Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1769vm-test-run-nixbot-gitlab> gitlab # [  304.690971] nixbot[3790]: {"ts": "2026-06-14T14:29:57.484Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1770vm-test-run-nixbot-gitlab> gitlab # [  304.693585] nixbot[3790]: {"ts": "2026-06-14T14:29:57.492Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1771vm-test-run-nixbot-gitlab> gitlab # [  304.695871] nixbot[3790]: {"ts": "2026-06-14T14:29:57.504Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1772vm-test-run-nixbot-gitlab> gitlab # [  304.697978] nixbot[3790]: {"ts": "2026-06-14T14:29:57.510Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1773vm-test-run-nixbot-gitlab> gitlab # [  304.700109] nixbot[3790]: {"ts": "2026-06-14T14:29:57.512Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1774vm-test-run-nixbot-gitlab> gitlab # [  304.702103] nixbot[3790]: {"ts": "2026-06-14T14:29:57.522Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1775vm-test-run-nixbot-gitlab> gitlab # [  304.704325] nixbot[3790]: {"ts": "2026-06-14T14:29:57.524Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1776vm-test-run-nixbot-gitlab> gitlab # [  304.706726] nixbot[3790]: {"ts": "2026-06-14T14:29:57.531Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1777vm-test-run-nixbot-gitlab> gitlab # [  304.708931] nixbot[3790]: {"ts": "2026-06-14T14:29:57.536Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1778vm-test-run-nixbot-gitlab> gitlab # [  304.711197] nixbot[3790]: {"ts": "2026-06-14T14:29:57.538Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1779vm-test-run-nixbot-gitlab> gitlab # [  304.713745] nixbot[3790]: {"ts": "2026-06-14T14:29:57.539Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1780vm-test-run-nixbot-gitlab> gitlab # [  304.716029] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.562Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"027f6985b73bdc039b2f648c","created_at":"2026-06-14T14:29:56.207Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.216Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-027f6985b73bdc039b2f648c: done: 0.112959 sec","job_status":"done","queue_duration_s":1.232845,"scheduling_latency_s":1.232871,"gitaly_calls":1,"gitaly_duration_s":0.029916,"redis_calls":15,"redis_duration_s":0.004742,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00081,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07799845999999999,"duration_s":0.112959,"completed_at":"2026-06-14T14:29:57.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1781vm-test-run-nixbot-gitlab> gitlab # [  304.761543] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.568Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"675b7b1cbba633cf76db869e","created_at":"2026-06-14T14:29:56.380Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.382Z","job_size_bytes":23,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-675b7b1cbba633cf76db869e: start","job_status":"start","queue_duration_s":1.185604,"scheduling_latency_s":1.185639}1782vm-test-run-nixbot-gitlab> gitlab # [  304.776399] gitaly[1356]: time="2026-06-14T14:29:57.509Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T20:29:57.480" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.481" grpc.time_ms=27.133 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783vm-test-run-nixbot-gitlab> gitlab # [  304.791126] nixbot[3790]: {"ts": "2026-06-14T14:29:57.542Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1784vm-test-run-nixbot-gitlab> gitlab # [  304.793217] nixbot[3790]: {"ts": "2026-06-14T14:29:57.544Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1785vm-test-run-nixbot-gitlab> gitlab # [  304.795096] nixbot[3790]: {"ts": "2026-06-14T14:29:57.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1786vm-test-run-nixbot-gitlab> gitlab # [  304.797183] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.644Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"675b7b1cbba633cf76db869e","created_at":"2026-06-14T14:29:56.380Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:56.382Z","job_size_bytes":23,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-675b7b1cbba633cf76db869e: done: 0.076382 sec","job_status":"done","queue_duration_s":1.185604,"scheduling_latency_s":1.185639,"gitaly_calls":1,"gitaly_duration_s":0.017643,"redis_calls":16,"redis_duration_s":0.007725,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000371,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.004339,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076382,"completed_at":"2026-06-14T14:29:57.644Z","load_balancing_strategy":"primary","db_duration_s":0.003749,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1787vm-test-run-nixbot-gitlab> gitlab # [  304.840311] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:57.646Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5a513561f93d8016ede0a953","created_at":"2026-06-14T14:29:57.134Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51dcef1af0f603655e0ef20cf66205ccd4e923010fccd42a93aad72cba86a7de","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:57.146Z","job_size_bytes":467,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5a513561f93d8016ede0a953: start","job_status":"start","queue_duration_s":0.499991,"scheduling_latency_s":0.500016}1788vm-test-run-nixbot-gitlab> gitlab # [  304.856565] gitaly[1356]: time="2026-06-14T14:29:57.631Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-14T14:30:27.215" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.616" grpc.time_ms=14.949 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789vm-test-run-nixbot-gitlab> gitlab # [  304.870099] gitaly[1356]: time="2026-06-14T14:29:57.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:27.309" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.710" grpc.time_ms=0.917 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790vm-test-run-nixbot-gitlab> gitlab # [  304.883361] gitaly[1356]: time="2026-06-14T14:29:57.729Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T14:30:07.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:57.714" grpc.time_ms=13.854 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791vm-test-run-nixbot-gitlab> gitlab # [  305.009895] gitaly[1356]: time="2026-06-14T14:29:57.910Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T14:30:27.507" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.908" grpc.time_ms=2.355 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792vm-test-run-nixbot-gitlab> gitlab # [  305.045877] gitaly[1356]: time="2026-06-14T14:29:57.926Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=295 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-14T14:30:27.514" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:57.915" grpc.time_ms=11.099 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793vm-test-run-nixbot-gitlab> gitlab # [  305.088692] nixbot[3790]: INFO:     Started server process [3790]1794vm-test-run-nixbot-gitlab> gitlab # [  305.090458] nixbot[3790]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1795vm-test-run-nixbot-gitlab> gitlab # [  305.092965] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50632 - "GET /health HTTP/1.1" 200 OK1796vm-test-run-nixbot-gitlab> gitlab # [  305.095727] systemd[1]: Started nixbot CI service.1797vm-test-run-nixbot-gitlab> gitlab # [  305.097072] systemd[1]: Reached target Multi-User System.1798vm-test-run-nixbot-gitlab> gitlab # [  305.098417] systemd[1]: Startup finished in 766ms (kernel) + 5.169s (initrd) + 4min 59.160s (userspace) = 5min 5.096s.1799vm-test-run-nixbot-gitlab> gitlab # [  305.177606] gitaly[1356]: time="2026-06-14T14:29:58.078Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:08.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:58.064" grpc.time_ms=13.286 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [  305.669065] gitaly[1356]: time="2026-06-14T14:29:58.569Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=884 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=9 component=gitaly.StreamServerInterceptor correlation_id=01KV38M4JN49K8PWDA68W2VWE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T14:30:28.045" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:58.545" grpc.time_ms=23.582 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.16 seconds)1802vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1803vm-test-run-nixbot-gitlab> gitlab # [  305.895196] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50640 - "GET /health HTTP/1.1" 200 OK1804vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1805vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1806vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1807vm-test-run-nixbot-gitlab> gitlab # [  306.030228] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 871808vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds)1809vm-test-run-nixbot-gitlab> []1810vm-test-run-nixbot-gitlab> gitlab # [  306.105773] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 10331811vm-test-run-nixbot-gitlab> gitlab # [  306.108484] gitaly[1356]: time="2026-06-14T14:29:59.006Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T14:30:28.582" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:58.983" grpc.time_ms=22.907 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812vm-test-run-nixbot-gitlab> gitlab # [  306.124566] nixbot[3790]: {"ts": "2026-06-14T14:29:59.009Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1813vm-test-run-nixbot-gitlab> gitlab # [  306.129568] gitaly[1356]: time="2026-06-14T14:29:59.030Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KV38M1WT4F6B1829DD7HNTNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-14T14:30:29.010" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:59.011" grpc.time_ms=18.322 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814vm-test-run-nixbot-gitlab> gitlab # [  306.143102] nixbot[3790]: {"ts": "2026-06-14T14:29:59.039Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1815vm-test-run-nixbot-gitlab> gitlab # [  306.197777] sidekiq-cluster[2418]: {"severity":"WARN","time":"2026-06-14T14:29:59.097Z","class":"CreatePipelineWorker","correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5e087cdad9dd54e78315af82c4122a879a9fec06","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5e087cdad9dd54e78315af82c4122a879a9fec06","ref":"refs/heads/master"},"retry":0}1816vm-test-run-nixbot-gitlab> gitlab # [  306.213972] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:59.113Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5a513561f93d8016ede0a953","created_at":"2026-06-14T14:29:57.134Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-a9be06d6c1e24237","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51dcef1af0f603655e0ef20cf66205ccd4e923010fccd42a93aad72cba86a7de","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:57.146Z","job_size_bytes":467,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5a513561f93d8016ede0a953: done: 1.467244 sec","job_status":"done","queue_duration_s":0.499991,"scheduling_latency_s":0.500016,"gitaly_calls":7,"gitaly_duration_s":0.109414,"redis_calls":53,"redis_duration_s":0.021043,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.00156,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.009688,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.0044,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003459,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.082,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.21492,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053648,"duration_s":1.467244,"completed_at":"2026-06-14T14:29:59.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.136191,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1817vm-test-run-nixbot-gitlab> gitlab # [  306.273963] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:59.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ae91539c61995f4164cc4938","created_at":"2026-06-14T14:29:57.423Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-7bf4a584e8564a2b","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:57.430Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ae91539c61995f4164cc4938: start","job_status":"start","queue_duration_s":1.686811,"scheduling_latency_s":1.686853}1818vm-test-run-nixbot-gitlab> gitlab # [  306.289663] nixbot[3790]: {"ts": "2026-06-14T14:29:59.124Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1819vm-test-run-nixbot-gitlab> gitlab # [  306.291734] nixbot[3790]: {"ts": "2026-06-14T14:29:59.127Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1820vm-test-run-nixbot-gitlab> gitlab # [  306.293716] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 711821vm-test-run-nixbot-gitlab> gitlab # [  306.295608] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:59.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ae91539c61995f4164cc4938","created_at":"2026-06-14T14:29:57.423Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-7bf4a584e8564a2b","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:57.430Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ae91539c61995f4164cc4938: done: 0.07574 sec","job_status":"done","queue_duration_s":1.686811,"scheduling_latency_s":1.686853,"redis_calls":19,"redis_duration_s":0.005295,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000921,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000796,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002648,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0131238,"duration_s":0.07574,"completed_at":"2026-06-14T14:29:59.193Z","load_balancing_strategy":"primary","db_duration_s":0.007723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1822vm-test-run-nixbot-gitlab> gitlab # [  306.338182] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:59.195Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ab0043479e5d7e7d2deef8db","created_at":"2026-06-14T14:29:59.170Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-c83d1d9a91454950","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:59.179Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ab0043479e5d7e7d2deef8db: start","job_status":"start","queue_duration_s":0.015764,"scheduling_latency_s":0.015788}1823vm-test-run-nixbot-gitlab> gitlab # [  306.416193] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:29:59.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ab0043479e5d7e7d2deef8db","created_at":"2026-06-14T14:29:59.170Z","trace_propagation_headers":{"sentry-trace":"c631ed9b726c4acb8eac6bd15e45ad87-c83d1d9a91454950","baggage":"sentry-trace_id=c631ed9b726c4acb8eac6bd15e45ad87,sentry-release=Unknown"},"correlation_id":"01KV38M1WT4F6B1829DD7HNTNJ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-14T14:29:59.179Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ab0043479e5d7e7d2deef8db: done: 0.121225 sec","job_status":"done","queue_duration_s":0.015764,"scheduling_latency_s":0.015788,"redis_calls":11,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002019,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037414,"duration_s":0.121225,"completed_at":"2026-06-14T14:29:59.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1824vm-test-run-nixbot-gitlab> gitlab # [  306.495558] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 2651825vm-test-run-nixbot-gitlab> gitlab # [  306.499015] nixbot[3790]: {"ts": "2026-06-14T14:29:59.398Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1826vm-test-run-nixbot-gitlab> gitlab # [  306.717962] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2061827vm-test-run-nixbot-gitlab> gitlab # [  306.724655] nixbot[3790]: {"ts": "2026-06-14T14:29:59.620Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1828vm-test-run-nixbot-gitlab> gitlab # [  306.839268] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 1051829vm-test-run-nixbot-gitlab> gitlab # [  306.841459] nixbot[3790]: {"ts": "2026-06-14T14:29:59.742Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1830vm-test-run-nixbot-gitlab> gitlab # [  306.914919] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:29:59 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 661831vm-test-run-nixbot-gitlab> gitlab # [  306.917588] nixbot[3790]: {"ts": "2026-06-14T14:29:59.817Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1832vm-test-run-nixbot-gitlab> gitlab # [  306.993035] gitaly[1356]: time="2026-06-14T14:29:59.893Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6CK2G5EWD3DKY8AGZV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-14T14:30:29.373" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:29:59.874" grpc.time_ms=18.333 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1834vm-test-run-nixbot-gitlab> gitlab # [  307.024209] gitaly[1356]: time="2026-06-14T14:29:59.924Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6CK2G5EWD3DKY8AGZV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:29.422" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:59.923" grpc.time_ms=1.265 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835vm-test-run-nixbot-gitlab> gitlab # [  307.060597] gitaly[1356]: time="2026-06-14T14:29:59.942Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6CK2G5EWD3DKY8AGZV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-14T14:30:09.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:29:59.931" grpc.time_ms=10.6 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1837vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-14T14:29:59.377Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1838vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.30 seconds)1839vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1840vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1841vm-test-run-nixbot-gitlab> gitlab # [  307.193373] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 881842vm-test-run-nixbot-gitlab> gitlab # [  307.213150] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 2851843vm-test-run-nixbot-gitlab> gitlab # [  307.220146] nixbot[3790]: {"ts": "2026-06-14T14:30:00.114Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1844vm-test-run-nixbot-gitlab> gitlab # [  307.322724] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 371845vm-test-run-nixbot-gitlab> gitlab # [  307.344968] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1251846vm-test-run-nixbot-gitlab> gitlab # [  307.353046] nixbot[3790]: {"ts": "2026-06-14T14:30:00.246Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1847vm-test-run-nixbot-gitlab> gitlab # [  307.361589] nixbot[3790]: {"ts": "2026-06-14T14:30:00.251Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1848vm-test-run-nixbot-gitlab> gitlab # [  307.451182] gitaly[1356]: time="2026-06-14T14:30:00.351Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6RZ3QNACTFXCGRH3MAA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.332" grpc.time_ms=18.781 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1849vm-test-run-nixbot-gitlab> gitlab # [  307.476676] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 1281850vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1851vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1852vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1853vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1854vm-test-run-nixbot-gitlab> gitlab # [  307.486189] gitaly[1356]: time="2026-06-14T14:30:00.386Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6VHA3H0Y1A65ARR0FQ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.377" grpc.time_ms=9.212 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1855vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)[  307.508498] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 821856vm-test-run-nixbot-gitlab> gitlab # Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 9.00 KiB/s, done.1857vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1858vm-test-run-nixbot-gitlab> gitlab # [  307.637829] gitaly[1356]: time="2026-06-14T14:30:00.538Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6YV7ANEMVV90WDJ60X4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.529" pid=1356 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1859vm-test-run-nixbot-gitlab> gitlab # [  307.649642] gitaly[1356]: time="2026-06-14T14:30:00.538Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6YV7ANEMVV90WDJ60X4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.529" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1860vm-test-run-nixbot-gitlab> gitlab # [  307.667402] gitlab-workhorse[1286]: time="2026-06-14T14:30:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491861vm-test-run-nixbot-gitlab> gitlab # [  307.670313] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1281862vm-test-run-nixbot-gitlab> gitlab # [  307.677072] gitaly[1356]: time="2026-06-14T14:30:00.539Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6YV7ANEMVV90WDJ60X4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.529" grpc.time_ms=9.56 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1863vm-test-run-nixbot-gitlab> gitlab # [  307.754530] gitaly[1356]: time="2026-06-14T14:30:00.655Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-14T14:30:10.113" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:00.643" grpc.time_ms=11.418 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864vm-test-run-nixbot-gitlab> gitlab # [  307.831631] gitaly[1356]: time="2026-06-14T14:30:00.732Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-14T14:30:30.119" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-14T14:30:00.720" grpc.time_ms=12.136 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865vm-test-run-nixbot-gitlab> gitlab # [  307.868094] gitaly[1356]: time="2026-06-14T14:30:00.768Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ duration_ms=190ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/allowed"1866vm-test-run-nixbot-gitlab> gitlab # [  307.913112] gitaly[1356]: time="2026-06-14T14:30:00.813Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ duration_ms=44ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/pre_receive"1867vm-test-run-nixbot-gitlab> gitlab # [  307.920068] gitaly[1356]: time="2026-06-14T14:30:00.817Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:00.577" grpc.time_ms=236.156 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1868vm-test-run-nixbot-gitlab> gitlab # [  307.944245] gitaly[1356]: time="2026-06-14T14:30:00.845Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:00.844" grpc.time_ms=0.626 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1869vm-test-run-nixbot-gitlab> gitlab # [  307.968524] gitaly[1356]: time="2026-06-14T14:30:00.869Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M72YR3MDKGZA5KEC5ETB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.838" pid=1356 remote_ip="2001:db8:1::1" request_sha=887a5c259ee6214d742c6ef86e42935f726f0cb7 response_bytes=378 span.kind=server system=grpc1870vm-test-run-nixbot-gitlab> gitlab # [  307.978467] gitaly[1356]: time="2026-06-14T14:30:00.869Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M72YR3MDKGZA5KEC5ETB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.838" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1871vm-test-run-nixbot-gitlab> gitlab # [  307.992258] gitlab-workhorse[1286]: time="2026-06-14T14:30:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3781872vm-test-run-nixbot-gitlab> gitlab # [  307.994255] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 378 "" "git/2.54.0" 3281873vm-test-run-nixbot-gitlab> gitlab # [  307.996752] gitaly[1356]: time="2026-06-14T14:30:00.869Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M72YR3MDKGZA5KEC5ETB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.838" grpc.time_ms=31.458 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1874vm-test-run-nixbot-gitlab> gitlab # [  308.010995] gitaly[1356]: time="2026-06-14T14:30:00.890Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:00.889" grpc.time_ms=0.466 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1875vm-test-run-nixbot-gitlab> gitlab # [  308.021533] gitaly[1356]: time="2026-06-14T14:30:00.922Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:00.921" grpc.time_ms=0.476 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1876vm-test-run-nixbot-gitlab> gitlab # [  308.105651] gitaly[1356]: time="2026-06-14T14:30:01.005Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M7F0VM6BDTW2K4AE9Z90 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.989" grpc.time_ms=16.553 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1877vm-test-run-nixbot-gitlab> gitlab # [  308.134468] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 771878vm-test-run-nixbot-gitlab> gitlab # [  308.137771] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.007Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5e087cdad9dd54e78315af82c4122a879a9fec06 da718fbc6918fb88595c136c01ea1539e2576084 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"052ec8c6746f274ffadd7e8b","created_at":"2026-06-14T14:30:01.001Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-e05ebb720e964ce4","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.004Z","job_size_bytes":185,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-052ec8c6746f274ffadd7e8b: start","job_status":"start","queue_duration_s":0.002719,"scheduling_latency_s":0.002761}1879vm-test-run-nixbot-gitlab> gitlab # [  308.154420] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.025Z","message":"changes: \"5e087cdad9dd54e78315af82c4122a879a9fec06 da718fbc6918fb88595c136c01ea1539e2576084 refs/heads/master\\n\""}1880vm-test-run-nixbot-gitlab> gitlab # [  308.165271] gitaly[1356]: time="2026-06-14T14:30:01.065Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=7 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:11.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.053" grpc.time_ms=12.314 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881vm-test-run-nixbot-gitlab> gitlab # [  308.180500] gitaly[1356]: time="2026-06-14T14:30:01.081Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ duration_ms=122ns method=POST pid=1356 status=200 url="http://unix/api/v4/internal/post_receive"1882vm-test-run-nixbot-gitlab> gitlab # [  308.183186] gitaly[1356]: time="2026-06-14T14:30:01.081Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:00.958" grpc.time_ms=123.182 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1883vm-test-run-nixbot-gitlab> gitlab # [  308.193871] gitaly[1356]: time="2026-06-14T14:30:01.091Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ diskcache=3e44b5be-d4c4-40bd-b823-34ada052ba59 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.489" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1884vm-test-run-nixbot-gitlab> gitlab # [  308.204035] gitlab-workhorse[1286]: time="2026-06-14T14:30:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521885vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1886vm-test-run-nixbot-gitlab> gitlab # [  308.205776] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:01 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6731887vm-test-run-nixbot-gitlab> gitlab #    5e087cd..da718fb  [  308.207918] gitaly[1356]: time="2026-06-14T14:30:01.091Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=151 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13072 command.oublock=32 command.real_time_ms=594 command.spawn_token_fork_ms=11 command.system_time_ms=98 command.user_time_ms=53 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:00.489" grpc.time_ms=602.115 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1888vm-test-run-nixbot-gitlab> gitlab # [  308.220769] gitaly[1356]: time="2026-06-14T14:30:01.098Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T14:30:11.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:01.071" grpc.time_ms=27.126 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889vm-test-run-nixbot-gitlab> gitlab # master -> master1890vm-test-run-nixbot-gitlab> gitlab # [  308.235229] gitaly[1356]: time="2026-06-14T14:30:01.136Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7HKBEGBVCZN3N9GRG27 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.128" pid=1356 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1891vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.06 seconds)1892vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1893vm-test-run-nixbot-gitlab> gitlab # [  308.245212] gitaly[1356]: time="2026-06-14T14:30:01.136Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7HKBEGBVCZN3N9GRG27 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.128" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1894vm-test-run-nixbot-gitlab> gitlab # [  308.257315] gitlab-workhorse[1286]: time="2026-06-14T14:30:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671895vm-test-run-nixbot-gitlab> gitlab # [  308.258879] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1251896vm-test-run-nixbot-gitlab> gitlab # [  308.260679] gitaly[1356]: time="2026-06-14T14:30:01.136Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7HKBEGBVCZN3N9GRG27 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.128" grpc.time_ms=8.137 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1897vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1898vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084'1899vm-test-run-nixbot-gitlab> gitlab # [  308.277315] gitaly[1356]: time="2026-06-14T14:30:01.140Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-14T14:30:11.064" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.124" grpc.time_ms=15.349 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900vm-test-run-nixbot-gitlab> gitlab # [  308.290343] gitaly[1356]: time="2026-06-14T14:30:01.157Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=8 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-14T14:30:31.145" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.145" grpc.time_ms=11.412 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901vm-test-run-nixbot-gitlab> gitlab # [  308.305463] gitaly[1356]: time="2026-06-14T14:30:01.181Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-14T14:30:11.111" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-14T14:30:01.171" grpc.time_ms=10.067 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902vm-test-run-nixbot-gitlab> gitlab # [  308.318058] gitaly[1356]: time="2026-06-14T14:30:01.198Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:11.128" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.189" grpc.time_ms=9.065 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084', in 0.06 seconds)1904vm-test-run-nixbot-gitlab> []1905vm-test-run-nixbot-gitlab> gitlab # [  308.330864] gitaly[1356]: time="2026-06-14T14:30:01.230Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7NK0P14TFWGTZK4MT3D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.200" pid=1356 remote_ip="2001:db8:1::1" request_sha=0477361f0f38c7bfdd5c4c3f712fab91ece16efd response_bytes=384 span.kind=server system=grpc1906vm-test-run-nixbot-gitlab> gitlab # [  308.341082] gitlab-workhorse[1286]: time="2026-06-14T14:30:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3841907vm-test-run-nixbot-gitlab> gitlab # [  308.342842] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 911908vm-test-run-nixbot-gitlab> gitlab # [  308.345077] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 200 OK1909vm-test-run-nixbot-gitlab> gitlab # [  308.347091] gitaly[1356]: time="2026-06-14T14:30:01.230Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1145 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=6 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7NK0P14TFWGTZK4MT3D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.200" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1910vm-test-run-nixbot-gitlab> gitlab # [  308.360226] gitaly[1356]: time="2026-06-14T14:30:01.230Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1145 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=6 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7NK0P14TFWGTZK4MT3D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:01.200" grpc.time_ms=30.773 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1911vm-test-run-nixbot-gitlab> gitlab # [  308.424798] gitaly[1356]: time="2026-06-14T14:30:01.325Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:31.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.324" grpc.time_ms=0.834 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912vm-test-run-nixbot-gitlab> gitlab # [  308.462857] gitaly[1356]: time="2026-06-14T14:30:01.363Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-14T14:30:31.054" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.354" grpc.time_ms=8.428 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913vm-test-run-nixbot-gitlab> gitlab # [  308.485923] gitaly[1356]: time="2026-06-14T14:30:01.386Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:11.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.372" grpc.time_ms=14.01 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914vm-test-run-nixbot-gitlab> gitlab # [  308.524973] gitaly[1356]: time="2026-06-14T14:30:01.425Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-14T14:30:11.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.424" grpc.time_ms=1.248 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915vm-test-run-nixbot-gitlab> gitlab # [  308.544883] gitaly[1356]: time="2026-06-14T14:30:01.445Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T14:30:11.074" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:01.445" grpc.time_ms=0.243 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916vm-test-run-nixbot-gitlab> gitlab # [  308.560644] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.455Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5e087cdad9dd54e78315af82c4122a879a9fec06 da718fbc6918fb88595c136c01ea1539e2576084 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"052ec8c6746f274ffadd7e8b","created_at":"2026-06-14T14:30:01.001Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-e05ebb720e964ce4","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.004Z","job_size_bytes":185,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-052ec8c6746f274ffadd7e8b: done: 0.447789 sec","job_status":"done","queue_duration_s":0.002719,"scheduling_latency_s":0.002761,"gitaly_calls":7,"gitaly_duration_s":0.106879,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035983,"redis_read_bytes":350,"redis_write_bytes":13064,"redis_cache_calls":9,"redis_cache_duration_s":0.004767,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001312,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6764,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.009897,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010383,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009131,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.253019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000794,"exclusive_lock_hold_duration_s":0.02366962,"duration_s":0.447789,"completed_at":"2026-06-14T14:30:01.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052316,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1917vm-test-run-nixbot-gitlab> gitlab # [  308.617253] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.457Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"35a409b25c009e0dd4fc653c","created_at":"2026-06-14T14:30:01.101Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.103Z","job_size_bytes":46,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-35a409b25c009e0dd4fc653c: start","job_status":"start","queue_duration_s":0.353293,"scheduling_latency_s":0.353321}1918vm-test-run-nixbot-gitlab> gitlab # [  308.632478] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.483Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"35a409b25c009e0dd4fc653c","created_at":"2026-06-14T14:30:01.101Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.103Z","job_size_bytes":46,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-35a409b25c009e0dd4fc653c: done: 0.026807 sec","job_status":"done","queue_duration_s":0.353293,"scheduling_latency_s":0.353321,"redis_calls":9,"redis_duration_s":0.004007,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00016,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026807,"completed_at":"2026-06-14T14:30:01.483Z","load_balancing_strategy":"primary","db_duration_s":0.002006,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1919vm-test-run-nixbot-gitlab> gitlab # [  308.675227] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.485Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7a353e9ad24aba82eb913779","created_at":"2026-06-14T14:30:01.112Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.121Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7a353e9ad24aba82eb913779: start","job_status":"start","queue_duration_s":0.363516,"scheduling_latency_s":0.363539}1920vm-test-run-nixbot-gitlab> gitlab # [  308.690746] gitaly[1356]: time="2026-06-14T14:30:01.462Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T14:30:11.082" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.453" grpc.time_ms=9.187 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921vm-test-run-nixbot-gitlab> gitlab # [  308.703055] gitaly[1356]: time="2026-06-14T14:30:01.523Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:01.522" grpc.time_ms=0.816 pid=1356 span.kind=server system=grpc1922vm-test-run-nixbot-gitlab> gitlab # [  308.712054] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.553Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7a353e9ad24aba82eb913779","created_at":"2026-06-14T14:30:01.112Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.121Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7a353e9ad24aba82eb913779: done: 0.067875 sec","job_status":"done","queue_duration_s":0.363516,"scheduling_latency_s":0.363539,"gitaly_calls":1,"gitaly_duration_s":0.03627,"redis_calls":12,"redis_duration_s":0.001926,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000329,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04971842,"duration_s":0.067875,"completed_at":"2026-06-14T14:30:01.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923vm-test-run-nixbot-gitlab> gitlab # [  308.754019] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:01.555Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9cc8dec6ed52960fef4da820","created_at":"2026-06-14T14:30:01.260Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0621e3265b2c90f3b9af1f29c144612ca0f97cb2efa90fadf394b75ab08bef65","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.263Z","job_size_bytes":467,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9cc8dec6ed52960fef4da820: start","job_status":"start","queue_duration_s":0.292266,"scheduling_latency_s":0.292287}1924vm-test-run-nixbot-gitlab> gitlab # [  308.770199] gitaly[1356]: time="2026-06-14T14:30:01.537Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ diskcache=d83b95a3-75c1-4eff-8ba9-4c5e6f1eecbb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T20:30:01.502" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.503" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925vm-test-run-nixbot-gitlab> gitlab # [  308.779476] gitaly[1356]: time="2026-06-14T14:30:01.537Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1509 command.oublock=0 command.real_time_ms=38 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T20:30:01.502" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.503" grpc.time_ms=34.33 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926vm-test-run-nixbot-gitlab> gitlab # [  308.793562] gitaly[1356]: time="2026-06-14T14:30:01.560Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:01.559" grpc.time_ms=0.349 pid=1356 span.kind=server system=grpc1927vm-test-run-nixbot-gitlab> gitlab # [  308.802554] gitaly[1356]: time="2026-06-14T14:30:01.568Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 diskcache=a4d43338-fa65-4638-a5b7-620916da4e82 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:11.094" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:01.464" pid=1356 ref=refs/keep-around/5e087cdad9dd54e78315af82c4122a879a9fec06 remote_ip="2001:db8:1::1" revision=5e087cdad9dd54e78315af82c4122a879a9fec06 span.kind=server system=grpc username=root1928vm-test-run-nixbot-gitlab> gitlab # [  308.813329] gitaly[1356]: time="2026-06-14T14:30:01.569Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=95 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5233 command.oublock=8 command.real_time_ms=97 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=53 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:11.094" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:01.464" grpc.time_ms=104.094 pid=1356 ref=refs/keep-around/5e087cdad9dd54e78315af82c4122a879a9fec06 remote_ip="2001:db8:1::1" revision=5e087cdad9dd54e78315af82c4122a879a9fec06 span.kind=server system=grpc username=root1929vm-test-run-nixbot-gitlab> gitlab # [  308.827208] gitaly[1356]: time="2026-06-14T14:30:01.572Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:31.271" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.572" grpc.time_ms=0.327 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930vm-test-run-nixbot-gitlab> gitlab # [  308.839291] gitaly[1356]: time="2026-06-14T14:30:01.584Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:31.583" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.584" grpc.time_ms=0.397 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [  308.851033] gitaly[1356]: time="2026-06-14T14:30:01.599Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T14:30:11.168" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.589" grpc.time_ms=10.204 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [  308.863317] nixbot[3790]: {"ts": "2026-06-14T14:30:01.750Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 \"HTTP/1.1 201 Created\""}1933vm-test-run-nixbot-gitlab> gitlab # [  308.867034] nixbot[3790]: {"ts": "2026-06-14T14:30:01.758Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5e087cdad9dd-60a9f560", "argv0": "bwrap"}1934vm-test-run-nixbot-gitlab> gitlab # [  308.869361] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4811935vm-test-run-nixbot-gitlab> gitlab # [  308.871687] gitaly[1356]: time="2026-06-14T14:30:01.636Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T14:30:31.635" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.635" grpc.time_ms=0.701 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936vm-test-run-nixbot-gitlab> gitlab # [  308.883718] gitaly[1356]: time="2026-06-14T14:30:01.644Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=22 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-14T14:30:31.642" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:01.642" grpc.time_ms=2.023 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [  308.898901] gitaly[1356]: time="2026-06-14T14:30:01.725Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=6 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:11.289" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:01.709" grpc.time_ms=15.661 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938vm-test-run-nixbot-gitlab> gitlab # [  309.064749] systemd[1]: Started Nix Daemon.1939vm-test-run-nixbot-gitlab> gitlab # [  309.134995] nix-daemon[4096]: accepted connection from pid 4094, user nixbot1940vm-test-run-nixbot-gitlab> gitlab # [  309.138050] nix-daemon[4096]: accepted connection from pid 4095, user nixbot1941vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084'1942vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084', in 0.09 seconds)1943vm-test-run-nixbot-gitlab> []1944vm-test-run-nixbot-gitlab> gitlab # [  309.386600] gitaly[1356]: time="2026-06-14T14:30:02.286Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T14:30:32.257" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:02.258" grpc.time_ms=26.673 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945vm-test-run-nixbot-gitlab> gitlab # [  309.430120] gitaly[1356]: time="2026-06-14T14:30:02.311Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M6Z2DYD0642NVZRGFBEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-14T14:30:32.288" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:02.289" grpc.time_ms=21.996 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946vm-test-run-nixbot-gitlab> gitlab # [  309.462988] sidekiq-cluster[2418]: {"severity":"WARN","time":"2026-06-14T14:30:02.328Z","class":"CreatePipelineWorker","correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5e087cdad9dd54e78315af82c4122a879a9fec06","after":"da718fbc6918fb88595c136c01ea1539e2576084","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"da718fbc6918fb88595c136c01ea1539e2576084","ref":"refs/heads/master"},"retry":0}1947vm-test-run-nixbot-gitlab> gitlab # [  309.476599] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.334Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9cc8dec6ed52960fef4da820","created_at":"2026-06-14T14:30:01.260Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0621e3265b2c90f3b9af1f29c144612ca0f97cb2efa90fadf394b75ab08bef65","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.263Z","job_size_bytes":467,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9cc8dec6ed52960fef4da820: done: 0.779132 sec","job_status":"done","queue_duration_s":0.292266,"scheduling_latency_s":0.292287,"gitaly_calls":7,"gitaly_duration_s":0.098988,"redis_calls":33,"redis_duration_s":0.00552,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000658,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002601,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.644733,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.779132,"completed_at":"2026-06-14T14:30:02.334Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038618,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1948vm-test-run-nixbot-gitlab> gitlab # [  309.544086] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.336Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"c1aacaa3-9b91-4ebc-aff6-a61b3c92d925\"}"],"class":"WebHookWorker","jid":"cc82e6d645082ab428ff5814","created_at":"2026-06-14T14:30:01.404Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6f348887ef97b305379edc74d4d65d58214f8020c0dbdae467c584d7bf97ff1e","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.410Z","job_size_bytes":1749,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-cc82e6d645082ab428ff5814: start","job_status":"start","queue_duration_s":0.925519,"scheduling_latency_s":0.925542}1949vm-test-run-nixbot-gitlab> gitlab # [  309.561962] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50658 - "GET /api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 200 OK1950vm-test-run-nixbot-gitlab> gitlab # [  309.652138] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50672 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1951vm-test-run-nixbot-gitlab> gitlab # [  309.671248] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.571Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"c1aacaa3-9b91-4ebc-aff6-a61b3c92d925\"}"],"class":"WebHookWorker","jid":"cc82e6d645082ab428ff5814","created_at":"2026-06-14T14:30:01.404Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-c66d86b775184392","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6f348887ef97b305379edc74d4d65d58214f8020c0dbdae467c584d7bf97ff1e","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.410Z","job_size_bytes":1749,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-cc82e6d645082ab428ff5814: done: 0.235117 sec","job_status":"done","queue_duration_s":0.925519,"scheduling_latency_s":0.925542,"redis_calls":19,"redis_duration_s":0.005207,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001624,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000512,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002059,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.015629619999970146,"cpu_s":0.206249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003757,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.235117,"completed_at":"2026-06-14T14:30:02.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1952vm-test-run-nixbot-gitlab> gitlab # [  309.765670] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.574Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f1a56c80c02e6e356ed2f196","created_at":"2026-06-14T14:30:01.577Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.583Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f1a56c80c02e6e356ed2f196: start","job_status":"start","queue_duration_s":0.990136,"scheduling_latency_s":0.990877}1953vm-test-run-nixbot-gitlab> gitlab # [  309.785199] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.675Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f1a56c80c02e6e356ed2f196","created_at":"2026-06-14T14:30:01.577Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.583Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f1a56c80c02e6e356ed2f196: done: 0.101561 sec","job_status":"done","queue_duration_s":0.990136,"scheduling_latency_s":0.990877,"redis_calls":18,"redis_duration_s":0.006958,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002721,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007031,"duration_s":0.101561,"completed_at":"2026-06-14T14:30:02.675Z","load_balancing_strategy":"primary","db_duration_s":0.022219,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1954vm-test-run-nixbot-gitlab> gitlab # [  309.834534] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.681Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9076bf162bf6d9a798641f29","created_at":"2026-06-14T14:30:01.584Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.590Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9076bf162bf6d9a798641f29: start","job_status":"start","queue_duration_s":1.091387,"scheduling_latency_s":1.091423}1955vm-test-run-nixbot-gitlab> gitlab # [  309.851551] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 701956vm-test-run-nixbot-gitlab> gitlab # [  309.853640] gitlab-workhorse[1286]: time="2026-06-14T14:30:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671957vm-test-run-nixbot-gitlab> gitlab # [  309.855366] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 711958vm-test-run-nixbot-gitlab> gitlab # [  309.857232] gitaly[1356]: time="2026-06-14T14:30:02.673Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M93B5WZ5XFQDZWEN09TB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:02.661" grpc.time_ms=11.571 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959vm-test-run-nixbot-gitlab> gitlab # [  309.869573] gitaly[1356]: time="2026-06-14T14:30:02.745Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M95KDPNV9G1V2K5KEEWY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:02.724" pid=1356 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1960vm-test-run-nixbot-gitlab> gitlab # [  309.879135] gitaly[1356]: time="2026-06-14T14:30:02.745Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M95KDPNV9G1V2K5KEEWY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:02.724" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1961vm-test-run-nixbot-gitlab> gitlab # [  309.891731] gitaly[1356]: time="2026-06-14T14:30:02.746Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M95KDPNV9G1V2K5KEEWY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:02.724" grpc.time_ms=21.774 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc1962vm-test-run-nixbot-gitlab> gitlab # [  309.907507] nixbot[3790]: {"ts": "2026-06-14T14:30:02.808Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1963vm-test-run-nixbot-gitlab> gitlab # [  309.916435] nix-daemon[4101]: unexpected Nix daemon error: error: interrupted by the user1964vm-test-run-nixbot-gitlab> gitlab # [  309.970431] gitaly[1356]: time="2026-06-14T14:30:02.871Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:32.569" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:02.870" grpc.time_ms=0.712 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965vm-test-run-nixbot-gitlab> gitlab # [  309.990785] gitaly[1356]: time="2026-06-14T14:30:02.891Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-14T14:30:32.580" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:02.880" grpc.time_ms=10.354 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966vm-test-run-nixbot-gitlab> gitlab # [  310.010282] gitaly[1356]: time="2026-06-14T14:30:02.911Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:12.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:02.897" grpc.time_ms=13.41 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967vm-test-run-nixbot-gitlab> gitlab # [  310.024856] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.917Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9076bf162bf6d9a798641f29","created_at":"2026-06-14T14:30:01.584Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.590Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9076bf162bf6d9a798641f29: done: 0.235402 sec","job_status":"done","queue_duration_s":1.091387,"scheduling_latency_s":1.091423,"redis_calls":21,"redis_duration_s":0.061057,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.008321,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.02146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.031276,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0019974,"duration_s":0.235402,"completed_at":"2026-06-14T14:30:02.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968vm-test-run-nixbot-gitlab> gitlab # [  310.069672] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:02.924Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0f58de096f36362fed8334f1","created_at":"2026-06-14T14:30:01.722Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.728Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0f58de096f36362fed8334f1: start","job_status":"start","queue_duration_s":1.196628,"scheduling_latency_s":1.196679}1969vm-test-run-nixbot-gitlab> gitlab # [  310.086669] gitaly[1356]: time="2026-06-14T14:30:02.937Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T14:30:12.045" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:02.936" grpc.time_ms=0.926 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970vm-test-run-nixbot-gitlab> gitlab # [  310.097280] gitaly[1356]: time="2026-06-14T14:30:02.949Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T14:30:12.051" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:02.941" grpc.time_ms=7.158 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971vm-test-run-nixbot-gitlab> gitlab # [  310.109755] gitaly[1356]: time="2026-06-14T14:30:02.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:02.995" grpc.time_ms=1.303 pid=1356 span.kind=server system=grpc1972vm-test-run-nixbot-gitlab> gitlab # [  310.129106] gitaly[1356]: time="2026-06-14T14:30:03.029Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:03.029" grpc.time_ms=0.684 pid=1356 span.kind=server system=grpc1973vm-test-run-nixbot-gitlab> gitlab # [  310.138921] gitaly[1356]: time="2026-06-14T14:30:03.034Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M9E9VZW5K9F42N0FBGMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:33.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:03.033" grpc.time_ms=1.019 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974vm-test-run-nixbot-gitlab> gitlab # [  310.151072] gitaly[1356]: time="2026-06-14T14:30:03.041Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ diskcache=76794535-7c9f-483f-8371-86a2a6d264a6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:12.060" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:02.951" pid=1356 ref=refs/keep-around/da718fbc6918fb88595c136c01ea1539e2576084 remote_ip="2001:db8:1::1" revision=da718fbc6918fb88595c136c01ea1539e2576084 span.kind=server system=grpc username=root1975vm-test-run-nixbot-gitlab> gitlab # [  310.161904] gitaly[1356]: time="2026-06-14T14:30:03.041Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4994 command.oublock=8 command.real_time_ms=82 command.spawn_token_fork_ms=3 command.system_time_ms=35 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T14:30:12.060" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T14:30:02.951" grpc.time_ms=90.391 pid=1356 ref=refs/keep-around/da718fbc6918fb88595c136c01ea1539e2576084 remote_ip="2001:db8:1::1" revision=da718fbc6918fb88595c136c01ea1539e2576084 span.kind=server system=grpc username=root1976vm-test-run-nixbot-gitlab> gitlab # [  310.176878] gitaly[1356]: time="2026-06-14T14:30:03.046Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:33.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:03.046" grpc.time_ms=0.319 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [  310.189768] redis[1250]: 10 changes in 300 seconds. Saving...1978vm-test-run-nixbot-gitlab> gitlab # [  310.190672] redis[1250]: Background saving started by pid 41631979vm-test-run-nixbot-gitlab> gitlab # [  310.191788] redis[4163]: BGSAVE done, 535 keys saved, 0 keys skipped, 133504 bytes written.1980vm-test-run-nixbot-gitlab> gitlab # [  310.193053] redis[4163]: DB saved on disk1981vm-test-run-nixbot-gitlab> gitlab # [  310.193728] redis[4163]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB1982vm-test-run-nixbot-gitlab> gitlab # [  310.234226] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3221983vm-test-run-nixbot-gitlab> gitlab # [  310.237869] nixbot[3790]: {"ts": "2026-06-14T14:30:03.136Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 \"HTTP/1.1 201 Created\""}1984vm-test-run-nixbot-gitlab> gitlab # [  310.240827] nixbot[3790]: {"ts": "2026-06-14T14:30:03.141Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-da718fbc6918-8ce423a3", "argv0": "bwrap"}1985vm-test-run-nixbot-gitlab> gitlab # [  310.265304] gitaly[1356]: time="2026-06-14T14:30:03.166Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M9E9VZW5K9F42N0FBGMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:13.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:03.157" grpc.time_ms=8.777 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986vm-test-run-nixbot-gitlab> gitlab # [  310.278464] redis[1250]: Background saving terminated with success1987vm-test-run-nixbot-gitlab> gitlab # [  310.340300] nix-daemon[4096]: accepted connection from pid 4174, user nixbot1988vm-test-run-nixbot-gitlab> gitlab # [  310.343881] nix-daemon[4096]: accepted connection from pid 4176, user nixbot1989vm-test-run-nixbot-gitlab> gitlab # [  310.382899] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 3301990vm-test-run-nixbot-gitlab> gitlab # [  310.385626] nixbot[3790]: {"ts": "2026-06-14T14:30:03.286Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 \"HTTP/1.1 201 Created\""}1991vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084'1992vm-test-run-nixbot-gitlab> gitlab # [  310.451409] gitaly[1356]: time="2026-06-14T14:30:03.351Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M9RXTMQQS0KXFYQ0Y3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:33.349" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:03.349" grpc.time_ms=0.894 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993vm-test-run-nixbot-gitlab> gitlab # [  310.489750] nixbot[3790]: INFO:     ::ffff:127.0.0.1:50676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 200 OK1994vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=da718fbc6918fb88595c136c01ea1539e2576084', in 0.07 seconds)1995vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-06-14T14:30:02.796605Z', 'started_at': '2026-06-14T14:30:03.137908Z', 'finished_at': None}]1996vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.30 seconds)1997vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses1998vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses1999vm-test-run-nixbot-gitlab> gitlab # [  310.500912] gitaly[1356]: time="2026-06-14T14:30:03.401Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV38M9RXTMQQS0KXFYQ0Y3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:13.241" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:03.391" grpc.time_ms=9.955 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2000vm-test-run-nixbot-gitlab> gitlab # [  310.568696] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1752001vm-test-run-nixbot-gitlab> gitlab # [  310.574622] nixbot[3790]: {"ts": "2026-06-14T14:30:03.470Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5e087cdad9dd54e78315af82c4122a879a9fec06 \"HTTP/1.1 201 Created\""}2002vm-test-run-nixbot-gitlab> gitlab # [  310.582854] gitaly[1356]: time="2026-06-14T14:30:03.483Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M9WXJJHD9GH7AC68TZWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:33.182" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:03.482" grpc.time_ms=1.047 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2003vm-test-run-nixbot-gitlab> gitlab # [  310.629229] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1082004vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.14 seconds)2005vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2006vm-test-run-nixbot-gitlab> gitlab # [  311.126226] gitaly[1356]: time="2026-06-14T14:30:04.026Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:04.025" grpc.time_ms=0.847 pid=1356 span.kind=server system=grpc2007vm-test-run-nixbot-gitlab> gitlab # [  311.182192] gitaly[1356]: time="2026-06-14T14:30:04.082Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:04.081" grpc.time_ms=0.935 pid=1356 span.kind=server system=grpc2008vm-test-run-nixbot-gitlab> gitlab # [  311.237617] gitaly[1356]: time="2026-06-14T14:30:04.138Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T14:30:04.137" grpc.time_ms=0.786 pid=1356 span.kind=server system=grpc2009vm-test-run-nixbot-gitlab> gitlab # [  311.262887] gitaly[1356]: time="2026-06-14T14:30:04.149Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 diskcache=cae764f3-fb1a-44e7-afbe-66481dba92e7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T14:30:33.065" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:03.966" pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010vm-test-run-nixbot-gitlab> gitlab # [  311.282413] gitaly[1356]: time="2026-06-14T14:30:04.150Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=191 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7290 command.oublock=0 command.real_time_ms=176 command.spawn_token_fork_ms=1 command.system_time_ms=111 command.user_time_ms=80 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T14:30:33.065" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:03.966" grpc.time_ms=183.463 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses2012vm-test-run-nixbot-gitlab> gitlab # [  311.769615] gitaly[1356]: time="2026-06-14T14:30:04.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MB1A0E4J7BP68SY4F3HY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:34.367" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:04.667" grpc.time_ms=1.096 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2013vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.19 seconds)2014vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2015vm-test-run-nixbot-gitlab> gitlab # [  311.821481] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1342016vm-test-run-nixbot-gitlab> gitlab # [  311.894945] gitaly[1356]: time="2026-06-14T14:30:04.795Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T14:30:34.793" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:04.793" grpc.time_ms=1.699 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2017vm-test-run-nixbot-gitlab> gitlab # [  312.131140] gitaly[1356]: time="2026-06-14T14:30:05.031Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:35.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:05.030" grpc.time_ms=1.349 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018vm-test-run-nixbot-gitlab> gitlab # [  312.167784] postgres[1328]: [1328] LOG:  checkpoint starting: time2019vm-test-run-nixbot-gitlab> gitlab # [  312.468466] gitaly[1356]: time="2026-06-14T14:30:05.368Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M7SK3WKMESR4FV8K41M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:35.366" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:05.367" grpc.time_ms=1.442 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020vm-test-run-nixbot-gitlab> gitlab # [  312.552881] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:05.451Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4940e0ead217b116751db29d","created_at":"2026-06-14T14:30:05.425Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-57bc20649c5c4f36","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"0f58de096f36362fed8334f1","job_size_bytes":3,"pid":2418,"job_status":"deduplicated","message":"PipelineProcessWorker JID-4940e0ead217b116751db29d: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2021vm-test-run-nixbot-gitlab> gitlab # [  312.592176] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:05.469Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0f58de096f36362fed8334f1","created_at":"2026-06-14T14:30:01.722Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.728Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2418,"job_status":"rescheduled","message":"PipelineProcessWorker JID-0f58de096f36362fed8334f1: rescheduled"}2022vm-test-run-nixbot-gitlab> gitlab # [  312.690725] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:05.589Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0f58de096f36362fed8334f1","created_at":"2026-06-14T14:30:01.722Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-6817a0a067e04d9b","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:01.728Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0f58de096f36362fed8334f1: done: 2.664613 sec","job_status":"done","queue_duration_s":1.196628,"scheduling_latency_s":1.196679,"gitaly_calls":4,"gitaly_duration_s":0.233085,"redis_calls":113,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.511674,"redis_read_bytes":410,"redis_write_bytes":25148,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.023549,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.029758,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":29,"redis_feature_flag_duration_s":0.215312,"redis_feature_flag_read_bytes":156,"redis_feature_flag_write_bytes":4863,"redis_queues_calls":22,"redis_queues_duration_s":0.057407,"redis_queues_read_bytes":33,"redis_queues_write_bytes":12382,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.091207,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3614,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.016525,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.011553,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.042814,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":850,"db_count":50,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":50,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":1.058,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.122,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.143,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.732,"db_ci_duration_s":0.327,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.541477,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":2.07382504,"duration_s":2.664613,"completed_at":"2026-06-14T14:30:05.589Z","load_balancing_strategy":"primary","db_duration_s":1.177096,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2023vm-test-run-nixbot-gitlab> gitlab # [  312.782390] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:05.605Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","cc82e6d645082ab428ff5814"],"class":"WebHooks::LogExecutionWorker","jid":"33dc84770eee2c5ce5dd2034","created_at":"2026-06-14T14:30:02.555Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-7c702c5c79214c37","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8f0088364f4a2bf696b157f20259c7e22485c66014abffe303d07e4a30aa7094","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:02.562Z","job_size_bytes":2506,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-33dc84770eee2c5ce5dd2034: start","job_status":"start","queue_duration_s":3.043337,"scheduling_latency_s":3.043388}2024vm-test-run-nixbot-gitlab> gitlab # [  312.800415] gitaly[1356]: time="2026-06-14T14:30:05.682Z" level=info msg="burden monitor stats" pid=1356 top_10_by_cpu="[]" top_10_by_memory="[]"2025vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses2026vm-test-run-nixbot-gitlab> gitlab # [  312.974680] gitaly[1356]: time="2026-06-14T14:30:05.875Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MC6M5XJBACHQ9PDN8M7K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:35.572" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:05.873" grpc.time_ms=1.278 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027vm-test-run-nixbot-gitlab> gitlab # [  313.030287] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1492028vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.21 seconds)2029vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2030vm-test-run-nixbot-gitlab> gitlab # [  313.342736] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.242Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","cc82e6d645082ab428ff5814"],"class":"WebHooks::LogExecutionWorker","jid":"33dc84770eee2c5ce5dd2034","created_at":"2026-06-14T14:30:02.555Z","trace_propagation_headers":{"sentry-trace":"73c03348a5224b549f6577c3782d15c2-7c702c5c79214c37","baggage":"sentry-trace_id=73c03348a5224b549f6577c3782d15c2,sentry-release=Unknown"},"correlation_id":"01KV38M6Z2DYD0642NVZRGFBEJ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8f0088364f4a2bf696b157f20259c7e22485c66014abffe303d07e4a30aa7094","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:02.562Z","job_size_bytes":2506,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-33dc84770eee2c5ce5dd2034: done: 0.636531 sec","job_status":"done","queue_duration_s":3.043337,"scheduling_latency_s":3.043388,"redis_calls":10,"redis_duration_s":0.03391,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00713,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007769,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.105,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.122,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.122,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.105,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.23864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00184442,"duration_s":0.636531,"completed_at":"2026-06-14T14:30:06.242Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.196121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031vm-test-run-nixbot-gitlab> gitlab # [  313.437587] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.252Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a5261d138b554ad8a3d7b2b6","created_at":"2026-06-14T14:30:03.049Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.053Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a5261d138b554ad8a3d7b2b6: start","job_status":"start","queue_duration_s":3.199476,"scheduling_latency_s":3.199526}2032vm-test-run-nixbot-gitlab> gitlab # [  313.494405] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.393Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a5261d138b554ad8a3d7b2b6","created_at":"2026-06-14T14:30:03.049Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.053Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a5261d138b554ad8a3d7b2b6: done: 0.141207 sec","job_status":"done","queue_duration_s":3.199476,"scheduling_latency_s":3.199526,"redis_calls":7,"redis_duration_s":0.019498,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006071,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.141207,"completed_at":"2026-06-14T14:30:06.393Z","load_balancing_strategy":"primary","db_duration_s":0.014745,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2033vm-test-run-nixbot-gitlab> gitlab # [  313.579249] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.406Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5516d44a84d025fb3d3a0368","created_at":"2026-06-14T14:30:03.054Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.057Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5516d44a84d025fb3d3a0368: start","job_status":"start","queue_duration_s":3.349162,"scheduling_latency_s":3.349213}2034vm-test-run-nixbot-gitlab> gitlab # [  313.742874] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.642Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5516d44a84d025fb3d3a0368","created_at":"2026-06-14T14:30:03.054Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.057Z","job_size_bytes":65,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5516d44a84d025fb3d3a0368: done: 0.235755 sec","job_status":"done","queue_duration_s":3.349162,"scheduling_latency_s":3.349213,"redis_calls":10,"redis_duration_s":0.021203,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.009395,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.235755,"completed_at":"2026-06-14T14:30:06.642Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035vm-test-run-nixbot-gitlab> gitlab # [  313.836988] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"715f0bdff082cf5a2eaf1c10","created_at":"2026-06-14T14:30:02.789Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e59143d6528be1991d48f9fc7dcbb7b1","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.074Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-715f0bdff082cf5a2eaf1c10: start","job_status":"start","queue_duration_s":3.580228,"scheduling_latency_s":3.580277}2036vm-test-run-nixbot-gitlab> gitlab # [  313.899207] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"715f0bdff082cf5a2eaf1c10","created_at":"2026-06-14T14:30:02.789Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e59143d6528be1991d48f9fc7dcbb7b1","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.074Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-715f0bdff082cf5a2eaf1c10: done: 0.143902 sec","job_status":"done","queue_duration_s":3.580228,"scheduling_latency_s":3.580277,"redis_calls":10,"redis_duration_s":0.023539,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001594,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00328402,"duration_s":0.143902,"completed_at":"2026-06-14T14:30:06.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037vm-test-run-nixbot-gitlab> gitlab # [  313.993030] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:06.811Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d4a73916a63f72f634549230","created_at":"2026-06-14T14:30:03.117Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.120Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d4a73916a63f72f634549230: start","job_status":"start","queue_duration_s":3.691012,"scheduling_latency_s":3.69106}2038vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses2039vm-test-run-nixbot-gitlab> gitlab # [  314.186980] gitaly[1356]: time="2026-06-14T14:30:07.086Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MDCPGNX46MSC3MQ4S6CT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:37.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:07.084" grpc.time_ms=1.304 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.21 seconds)2041vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2042vm-test-run-nixbot-gitlab> gitlab # [  314.220468] gitaly[1356]: time="2026-06-14T14:30:07.092Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T14:30:37.089" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:07.090" grpc.time_ms=2.406 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043vm-test-run-nixbot-gitlab> gitlab # [  314.243552] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1382044vm-test-run-nixbot-gitlab> gitlab # [  314.321952] gitaly[1356]: time="2026-06-14T14:30:07.222Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:37.220" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:07.221" grpc.time_ms=1.224 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045vm-test-run-nixbot-gitlab> gitlab # [  314.397970] gitaly[1356]: time="2026-06-14T14:30:07.298Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38M99WMQVKP6GZ9RT24ADZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:37.296" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:07.296" grpc.time_ms=1.333 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046vm-test-run-nixbot-gitlab> gitlab # [  314.449628] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.349Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d4a73916a63f72f634549230","created_at":"2026-06-14T14:30:03.117Z","trace_propagation_headers":{"sentry-trace":"a3b9c7f720434ccc9f5c9b7a978bedde-5a7840692a4542b7","baggage":"sentry-trace_id=a3b9c7f720434ccc9f5c9b7a978bedde,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV38M99WMQVKP6GZ9RT24ADZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.120Z","job_size_bytes":3,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d4a73916a63f72f634549230: done: 0.53778 sec","job_status":"done","queue_duration_s":3.691012,"scheduling_latency_s":3.69106,"gitaly_calls":3,"gitaly_duration_s":0.069864,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.06611,"redis_read_bytes":99,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.009992,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.011097,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.001413,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.017531,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.003601,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.005533,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006951,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.116,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.02,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.046,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.245304,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.40625354,"duration_s":0.53778,"completed_at":"2026-06-14T14:30:07.349Z","load_balancing_strategy":"primary","db_duration_s":0.132751,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2047vm-test-run-nixbot-gitlab> gitlab # [  314.553345] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"a4e6ddf40f829ec0a0f4650b","created_at":"2026-06-14T14:30:03.118Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9fc58b5dbd8c90f13e10c59cc1dd796","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.156Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-a4e6ddf40f829ec0a0f4650b: start","job_status":"start","queue_duration_s":4.204907,"scheduling_latency_s":4.204958}2048vm-test-run-nixbot-gitlab> gitlab # [  314.685308] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"a4e6ddf40f829ec0a0f4650b","created_at":"2026-06-14T14:30:03.118Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9fc58b5dbd8c90f13e10c59cc1dd796","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.156Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-a4e6ddf40f829ec0a0f4650b: done: 0.222008 sec","job_status":"done","queue_duration_s":4.204907,"scheduling_latency_s":4.204958,"redis_calls":26,"redis_duration_s":0.057464,"redis_read_bytes":27,"redis_write_bytes":6237,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.012928,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":969,"redis_queues_calls":2,"redis_queues_duration_s":0.003168,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3805,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.024487,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.016881,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00331318,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.222008,"completed_at":"2026-06-14T14:30:07.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049vm-test-run-nixbot-gitlab> gitlab # [  314.787634] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d7a6354398670bbe31fc5f2e","created_at":"2026-06-14T14:30:03.244Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45dc28c4f7ef6e57fd60416f72539406","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.286Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d7a6354398670bbe31fc5f2e: start","job_status":"start","queue_duration_s":4.309686,"scheduling_latency_s":4.309734}2050vm-test-run-nixbot-gitlab> gitlab # [  314.828604] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d7a6354398670bbe31fc5f2e","created_at":"2026-06-14T14:30:03.244Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45dc28c4f7ef6e57fd60416f72539406","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.286Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d7a6354398670bbe31fc5f2e: done: 0.131697 sec","job_status":"done","queue_duration_s":4.309686,"scheduling_latency_s":4.309734,"redis_calls":9,"redis_duration_s":0.021901999999999998,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004169,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005957,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.131697,"completed_at":"2026-06-14T14:30:07.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051vm-test-run-nixbot-gitlab> gitlab # [  314.903879] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b26fb9b3aae8e6c44a863db6","created_at":"2026-06-14T14:30:03.296Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1898bb0857dab29384a8d7c89de1935b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.315Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b26fb9b3aae8e6c44a863db6: start","job_status":"start","queue_duration_s":4.422932,"scheduling_latency_s":4.422982}2052vm-test-run-nixbot-gitlab> gitlab # [  314.980780] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b26fb9b3aae8e6c44a863db6","created_at":"2026-06-14T14:30:03.296Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1898bb0857dab29384a8d7c89de1935b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.315Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b26fb9b3aae8e6c44a863db6: done: 0.141511 sec","job_status":"done","queue_duration_s":4.422932,"scheduling_latency_s":4.422982,"redis_calls":13,"redis_duration_s":0.034054,"redis_read_bytes":11,"redis_write_bytes":1663,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.018561,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":887,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.141511,"completed_at":"2026-06-14T14:30:07.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053vm-test-run-nixbot-gitlab> gitlab # [  315.066240] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:07.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1892d10e5cfdef61e2514ae1","created_at":"2026-06-14T14:30:03.318Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f80ed6a5d0f1761650891dbce29cb49","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.347Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1892d10e5cfdef61e2514ae1: start","job_status":"start","queue_duration_s":4.544007,"scheduling_latency_s":4.544056}2054vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses2055vm-test-run-nixbot-gitlab> gitlab # [  315.174040] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1892d10e5cfdef61e2514ae1","created_at":"2026-06-14T14:30:03.318Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f80ed6a5d0f1761650891dbce29cb49","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.347Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1892d10e5cfdef61e2514ae1: done: 0.182182 sec","job_status":"done","queue_duration_s":4.544007,"scheduling_latency_s":4.544056,"redis_calls":18,"redis_duration_s":0.043561,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00844,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.005735,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.022175,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.007211,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.182182,"completed_at":"2026-06-14T14:30:08.073Z","load_balancing_strategy":"primary","db_duration_s":0.008785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2056vm-test-run-nixbot-gitlab> gitlab # [  315.270921] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1ae1b4125408475ddb5796a6","created_at":"2026-06-14T14:30:03.353Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18618a8de37d931dd3a6ee473fb9532f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.377Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1ae1b4125408475ddb5796a6: start","job_status":"start","queue_duration_s":4.708633,"scheduling_latency_s":4.708679}2057vm-test-run-nixbot-gitlab> gitlab # [  315.372854] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1ae1b4125408475ddb5796a6","created_at":"2026-06-14T14:30:03.353Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18618a8de37d931dd3a6ee473fb9532f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.377Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1ae1b4125408475ddb5796a6: done: 0.184585 sec","job_status":"done","queue_duration_s":4.708633,"scheduling_latency_s":4.708679,"redis_calls":10,"redis_duration_s":0.015861,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003849,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00351692,"duration_s":0.184585,"completed_at":"2026-06-14T14:30:08.270Z","load_balancing_strategy":"primary","db_duration_s":0.039073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058vm-test-run-nixbot-gitlab> gitlab # [  315.450456] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"1ad81ca377a52d4f82e356f3","created_at":"2026-06-14T14:30:03.417Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"302b67ed0205cb0250de1729197119f6","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.451Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-1ad81ca377a52d4f82e356f3: start","job_status":"start","queue_duration_s":4.830013,"scheduling_latency_s":4.830072}2059vm-test-run-nixbot-gitlab> gitlab # [  315.467246] gitaly[1356]: time="2026-06-14T14:30:08.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MEJXJMFJXN43JPRMJAP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:38.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:08.315" grpc.time_ms=6.15 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.25 seconds)2061vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2062vm-test-run-nixbot-gitlab> gitlab # [  315.481028] gitaly[1356]: time="2026-06-14T14:30:08.373Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MEMH4MM3H8ADHP856848 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:38.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:08.370" grpc.time_ms=2.887 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063vm-test-run-nixbot-gitlab> gitlab # [  315.493437] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1632064vm-test-run-nixbot-gitlab> gitlab # [  315.512131] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"1ad81ca377a52d4f82e356f3","created_at":"2026-06-14T14:30:03.417Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"302b67ed0205cb0250de1729197119f6","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.451Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-1ad81ca377a52d4f82e356f3: done: 0.131012 sec","job_status":"done","queue_duration_s":4.830013,"scheduling_latency_s":4.830072,"redis_calls":13,"redis_duration_s":0.002371,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000562,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000859,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029792,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.131012,"completed_at":"2026-06-14T14:30:08.412Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065vm-test-run-nixbot-gitlab> gitlab # [  315.575244] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"0875a1feb0f218c3490fe6c9","created_at":"2026-06-14T14:30:03.458Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68dfc2fd161e6a7113241f44680d98e8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.480Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-0875a1feb0f218c3490fe6c9: start","job_status":"start","queue_duration_s":4.93293,"scheduling_latency_s":4.93295}2066vm-test-run-nixbot-gitlab> gitlab # [  315.591526] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"0875a1feb0f218c3490fe6c9","created_at":"2026-06-14T14:30:03.458Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68dfc2fd161e6a7113241f44680d98e8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.480Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-0875a1feb0f218c3490fe6c9: done: 0.028882 sec","job_status":"done","queue_duration_s":4.93293,"scheduling_latency_s":4.93295,"redis_calls":11,"redis_duration_s":0.002447,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0002,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003092,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.028882,"completed_at":"2026-06-14T14:30:08.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2067vm-test-run-nixbot-gitlab> gitlab # [  315.635117] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a25c8e88e21727559a99ff7e","created_at":"2026-06-14T14:30:03.486Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55c4d8e7f6422618d2769d67344606fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.515Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a25c8e88e21727559a99ff7e: start","job_status":"start","queue_duration_s":4.928742,"scheduling_latency_s":4.928763}2068vm-test-run-nixbot-gitlab> gitlab # [  315.648215] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a25c8e88e21727559a99ff7e","created_at":"2026-06-14T14:30:03.486Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55c4d8e7f6422618d2769d67344606fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.515Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a25c8e88e21727559a99ff7e: done: 0.021837 sec","job_status":"done","queue_duration_s":4.928742,"scheduling_latency_s":4.928763,"redis_calls":11,"redis_duration_s":0.001882,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000307,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000714,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056546,"exclusive_lock_hold_duration_s":0.00273426,"duration_s":0.021837,"completed_at":"2026-06-14T14:30:08.466Z","load_balancing_strategy":"primary","db_duration_s":0.001124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069vm-test-run-nixbot-gitlab> gitlab # [  315.687742] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"f683b05ae21e8af65fd291a9","created_at":"2026-06-14T14:30:03.545Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74379f3ddf7d351a747d41e115ad65ff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.585Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-f683b05ae21e8af65fd291a9: start","job_status":"start","queue_duration_s":4.882536,"scheduling_latency_s":4.882555}2070vm-test-run-nixbot-gitlab> gitlab # [  315.701794] gitaly[1356]: time="2026-06-14T14:30:08.426Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38MEMH4MM3H8ADHP856848 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:18.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:08.407" grpc.time_ms=18.41 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071vm-test-run-nixbot-gitlab> gitlab # [  315.715052] nixbot[3790]: {"ts": "2026-06-14T14:30:08.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 \"HTTP/1.1 201 Created\""}2072vm-test-run-nixbot-gitlab> gitlab # [  315.717807] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"f683b05ae21e8af65fd291a9","created_at":"2026-06-14T14:30:03.545Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74379f3ddf7d351a747d41e115ad65ff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.585Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-f683b05ae21e8af65fd291a9: done: 0.018731 sec","job_status":"done","queue_duration_s":4.882536,"scheduling_latency_s":4.882555,"redis_calls":12,"redis_duration_s":0.0017980000000000001,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002148,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.018731,"completed_at":"2026-06-14T14:30:08.486Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2073vm-test-run-nixbot-gitlab> gitlab # [  315.761995] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5d264c472e3aad71c69b5efa","created_at":"2026-06-14T14:30:03.613Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3eab06212fd748dbfe4518e76b49072a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.663Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5d264c472e3aad71c69b5efa: start","job_status":"start","queue_duration_s":4.824161,"scheduling_latency_s":4.824186}2074vm-test-run-nixbot-gitlab> gitlab # [  315.775864] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5d264c472e3aad71c69b5efa","created_at":"2026-06-14T14:30:03.613Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3eab06212fd748dbfe4518e76b49072a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.663Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5d264c472e3aad71c69b5efa: done: 0.020302 sec","job_status":"done","queue_duration_s":4.824161,"scheduling_latency_s":4.824186,"redis_calls":11,"redis_duration_s":0.001964,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005554,"exclusive_lock_hold_duration_s":0.0020512,"duration_s":0.020302,"completed_at":"2026-06-14T14:30:08.508Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075vm-test-run-nixbot-gitlab> gitlab # [  315.816495] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.509Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d043033c8cd73a65d901f391","created_at":"2026-06-14T14:30:03.689Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfedeb9d34230aa911093def18100d97","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.744Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d043033c8cd73a65d901f391: start","job_status":"start","queue_duration_s":4.765547,"scheduling_latency_s":4.765571}2076vm-test-run-nixbot-gitlab> gitlab # [  315.830010] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d043033c8cd73a65d901f391","created_at":"2026-06-14T14:30:03.689Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfedeb9d34230aa911093def18100d97","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.744Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d043033c8cd73a65d901f391: done: 0.018623 sec","job_status":"done","queue_duration_s":4.765547,"scheduling_latency_s":4.765571,"redis_calls":11,"redis_duration_s":0.001572,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000608,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004814,"exclusive_lock_hold_duration_s":0.00256894,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.018623,"completed_at":"2026-06-14T14:30:08.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2077vm-test-run-nixbot-gitlab> gitlab # [  315.871498] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"86ed775e3fb6bd4f1e5f8d6a","created_at":"2026-06-14T14:30:03.777Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35cfa8d958abc5308f483e83612d9af8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.820Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-86ed775e3fb6bd4f1e5f8d6a: start","job_status":"start","queue_duration_s":4.709055,"scheduling_latency_s":4.709072}2078vm-test-run-nixbot-gitlab> gitlab # [  315.885615] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 2032079vm-test-run-nixbot-gitlab> gitlab # [  315.887979] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2532080vm-test-run-nixbot-gitlab> gitlab # [  315.890578] gitaly[1356]: time="2026-06-14T14:30:08.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MEV8SA1RZ1XE4Z7DV7YC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:38.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:08.592" grpc.time_ms=1.049 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081vm-test-run-nixbot-gitlab> gitlab # [  315.902684] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"86ed775e3fb6bd4f1e5f8d6a","created_at":"2026-06-14T14:30:03.777Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35cfa8d958abc5308f483e83612d9af8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.820Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-86ed775e3fb6bd4f1e5f8d6a: done: 0.177857 sec","job_status":"done","queue_duration_s":4.709055,"scheduling_latency_s":4.709072,"redis_calls":20,"redis_duration_s":0.003576,"redis_read_bytes":23,"redis_write_bytes":4315,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000884,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000768,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2020,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033478,"duration_s":0.177857,"completed_at":"2026-06-14T14:30:08.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082vm-test-run-nixbot-gitlab> gitlab # [  315.945600] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b23eb72b5413b22be096613e","created_at":"2026-06-14T14:30:03.797Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-57bc20649c5c4f36","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.834Z","job_size_bytes":23,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b23eb72b5413b22be096613e: start","job_status":"start","queue_duration_s":4.873908,"scheduling_latency_s":4.873927}2083vm-test-run-nixbot-gitlab> gitlab # [  315.961477] nixbot[3790]: {"ts": "2026-06-14T14:30:08.743Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 \"HTTP/1.1 201 Created\""}2084vm-test-run-nixbot-gitlab> gitlab # [  315.963997] gitaly[1356]: time="2026-06-14T14:30:08.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38MEV8SA1RZ1XE4Z7DV7YC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:18.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:08.651" grpc.time_ms=13.896 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085vm-test-run-nixbot-gitlab> gitlab # [  315.976648] gitaly[1356]: time="2026-06-14T14:30:08.815Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV38MF3JYKV68HD6HJFJYDJE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T14:30:38.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T14:30:08.814" grpc.time_ms=0.666 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2086vm-test-run-nixbot-gitlab> gitlab # [  315.988767] gitaly[1356]: time="2026-06-14T14:30:08.865Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KV38MF3JYKV68HD6HJFJYDJE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T14:30:18.213" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T14:30:08.854" grpc.time_ms=10.365 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087vm-test-run-nixbot-gitlab> gitlab # [  316.026821] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1722088vm-test-run-nixbot-gitlab> gitlab # [  316.029461] nixbot[3790]: {"ts": "2026-06-14T14:30:08.928Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da718fbc6918fb88595c136c01ea1539e2576084 \"HTTP/1.1 201 Created\""}2089vm-test-run-nixbot-gitlab> gitlab # [  316.032266] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b23eb72b5413b22be096613e","created_at":"2026-06-14T14:30:03.797Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-57bc20649c5c4f36","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.834Z","job_size_bytes":23,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b23eb72b5413b22be096613e: done: 0.220236 sec","job_status":"done","queue_duration_s":4.873908,"scheduling_latency_s":4.873927,"redis_calls":15,"redis_duration_s":0.004815,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000408,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002352,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000756,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185508,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056468,"duration_s":0.220236,"completed_at":"2026-06-14T14:30:08.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024995,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2090vm-test-run-nixbot-gitlab> gitlab # [  316.077983] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:08.932Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"6b2dbec4306e3e64dd6cc1ba","created_at":"2026-06-14T14:30:08.911Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-f20cccb1b1664e98","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:dd07e597c7f5ca028fb2fdf7fcefff9c54926f58a996b66b8bec64427a389f66","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:08.915Z","job_size_bytes":464,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-6b2dbec4306e3e64dd6cc1ba: start","job_status":"start","queue_duration_s":0.016895,"scheduling_latency_s":0.016934}2091vm-test-run-nixbot-gitlab> gitlab # [  316.106312] sidekiq-cluster[2418]: {"severity":"WARN","time":"2026-06-14T14:30:09.005Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"6b2dbec4306e3e64dd6cc1ba","created_at":"2026-06-14T14:30:08.911Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-f20cccb1b1664e98","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_id":"01KV38M7SK3WKMESR4FV8K41M8","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:dd07e597c7f5ca028fb2fdf7fcefff9c54926f58a996b66b8bec64427a389f66","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:08.915Z","job_size_bytes":464,"pid":2418,"sidekiq_tid":"m4fa","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-6b2dbec4306e3e64dd6cc1ba: fail: 0.071443 sec","job_status":"fail","queue_duration_s":0.016895,"scheduling_latency_s":0.016934,"redis_calls":7,"redis_duration_s":0.002415,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000638,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071443,"completed_at":"2026-06-14T14:30:09.003Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.01076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092vm-test-run-nixbot-gitlab> gitlab # [  316.471370] nix-daemon[4096]: accepted connection from pid 4261, user nixbot2093vm-test-run-nixbot-gitlab> gitlab # [  316.472315] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 702094vm-test-run-nixbot-gitlab> gitlab # [  316.474370] gitaly[1356]: time="2026-06-14T14:30:09.313Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV38MFJV04DZSRG1SWD8S3GN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:09.302" grpc.time_ms=10.142 pid=1356 remote_ip="2001:db8:1::1" span.kind=server system=grpc2095vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses2096vm-test-run-nixbot-gitlab> gitlab # [  316.486770] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e677088aaa1c253ba17a9e98","created_at":"2026-06-14T14:30:03.854Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86ebd3db2dd704cbcd3e7600dfb205db","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.897Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e677088aaa1c253ba17a9e98: start","job_status":"start","queue_duration_s":5.113179,"scheduling_latency_s":5.113213}2097vm-test-run-nixbot-gitlab> gitlab # [  316.498938] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e677088aaa1c253ba17a9e98","created_at":"2026-06-14T14:30:03.854Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86ebd3db2dd704cbcd3e7600dfb205db","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.897Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e677088aaa1c253ba17a9e98: done: 0.037887 sec","job_status":"done","queue_duration_s":5.113179,"scheduling_latency_s":5.113213,"redis_calls":10,"redis_duration_s":0.003023,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000699,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000626,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047358,"duration_s":0.037887,"completed_at":"2026-06-14T14:30:09.047Z","load_balancing_strategy":"primary","db_duration_s":0.001082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098vm-test-run-nixbot-gitlab> gitlab # [  316.536054] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f6b875299ded0ff38f23dfac","created_at":"2026-06-14T14:30:03.945Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f90c6bdb7eef298c0b1aeed46d287268","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.975Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f6b875299ded0ff38f23dfac: start","job_status":"start","queue_duration_s":5.074662,"scheduling_latency_s":5.074686}2099vm-test-run-nixbot-gitlab> gitlab # [  316.549165] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f6b875299ded0ff38f23dfac","created_at":"2026-06-14T14:30:03.945Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f90c6bdb7eef298c0b1aeed46d287268","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:03.975Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f6b875299ded0ff38f23dfac: done: 0.036094 sec","job_status":"done","queue_duration_s":5.074662,"scheduling_latency_s":5.074686,"redis_calls":9,"redis_duration_s":0.002689,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036094,"completed_at":"2026-06-14T14:30:09.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100vm-test-run-nixbot-gitlab> gitlab # [  316.590413] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d24863fdb47c4c8dbe23aaf6","created_at":"2026-06-14T14:30:03.998Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef040e4360d3ae98219f12a979be8546","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.049Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d24863fdb47c4c8dbe23aaf6: start","job_status":"start","queue_duration_s":5.038617,"scheduling_latency_s":5.038638}2101vm-test-run-nixbot-gitlab> gitlab # [  316.603659] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d24863fdb47c4c8dbe23aaf6","created_at":"2026-06-14T14:30:03.998Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef040e4360d3ae98219f12a979be8546","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.049Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d24863fdb47c4c8dbe23aaf6: done: 0.410335 sec","job_status":"done","queue_duration_s":5.038617,"scheduling_latency_s":5.038638,"redis_calls":573,"redis_duration_s":0.083957,"redis_read_bytes":571,"redis_write_bytes":49790,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.082452,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.359948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.410335,"completed_at":"2026-06-14T14:30:09.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102vm-test-run-nixbot-gitlab> gitlab # [  316.646002] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"9151c3c4b25d118ffd606921","created_at":"2026-06-14T14:30:04.068Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e140c4b8226d0c47fea0bc415aed8848","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.095Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-9151c3c4b25d118ffd606921: start","job_status":"start","queue_duration_s":5.406959,"scheduling_latency_s":5.40699}2103vm-test-run-nixbot-gitlab> gitlab # [  316.658669] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"9151c3c4b25d118ffd606921","created_at":"2026-06-14T14:30:04.068Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e140c4b8226d0c47fea0bc415aed8848","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.095Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-9151c3c4b25d118ffd606921: done: 0.022448 sec","job_status":"done","queue_duration_s":5.406959,"scheduling_latency_s":5.40699,"redis_calls":11,"redis_duration_s":0.0023439999999999997,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028264,"duration_s":0.022448,"completed_at":"2026-06-14T14:30:09.524Z","load_balancing_strategy":"primary","db_duration_s":0.00056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104vm-test-run-nixbot-gitlab> gitlab # [  316.697616] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d78148df1c33c4e3d5dec856","created_at":"2026-06-14T14:30:04.111Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"258e5c9b88de6465dafba561ffd7d684","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.149Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d78148df1c33c4e3d5dec856: start","job_status":"start","queue_duration_s":5.377372,"scheduling_latency_s":5.377394}2105vm-test-run-nixbot-gitlab> gitlab # [  316.711880] gitlab-workhorse[1286]: time="2026-06-14T14:30:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672106vm-test-run-nixbot-gitlab> gitlab # [  316.713762] gitlab-workhorse[1286]: gitlab 2001:db8:1::1 - - [2026/06/14:14:30:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 2372107vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/da718fbc6918fb88595c136c01ea1539e2576084/statuses, in 0.24 seconds)2108vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-14T14:30:03.111Z', 'started_at': None, 'finished_at': '2026-06-14T14:30:08.450Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'da718fbc6918fb88595c136c01ea1539e2576084', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-06-14T14:30:08.708Z', 'started_at': None, 'finished_at': '2026-06-14T14:30:08.897Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2109vm-test-run-nixbot-gitlab> gitlab # [  316.715667] gitaly[1356]: time="2026-06-14T14:30:09.552Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV38MFN3A6G6JTWS32R7ACR4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T14:30:09.525" pid=1356 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2110vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 6.24 seconds)2111vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 379.70 seconds)2112vm-test-run-nixbot-gitlab> gitlab # [  316.725683] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d78148df1c33c4e3d5dec856","created_at":"2026-06-14T14:30:04.111Z","trace_propagation_headers":{"sentry-trace":"ed5561eabc1b47f78f67f72efbabc6e1-3c6e6e13b7e743b5","baggage":"sentry-trace_id=ed5561eabc1b47f78f67f72efbabc6e1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"258e5c9b88de6465dafba561ffd7d684","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T14:30:04.149Z","job_size_bytes":2,"pid":2418,"sidekiq_tid":"maby","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d78148df1c33c4e3d5dec856: done: 0.044114 sec","job_status":"done","queue_duration_s":5.377372,"scheduling_latency_s":5.377394,"redis_calls":11,"redis_duration_s":0.001816,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00065648,"exclusive_lock_hold_duration_s":0.01576396,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.044114,"completed_at":"2026-06-14T14:30:09.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113vm-test-run-nixbot-gitlab> test script finished in 379.75s2114vm-test-run-nixbot-gitlab> cleanup2115vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2116vm-test-run-nixbot-gitlab> gitlab # [  316.777008] sidekiq-cluster[2418]: {"severity":"INFO","time":"2026-06-14T14:30:09.573Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b14ec2268a3c9ec1b570f3d5","created_at":"2026-06-14T14:30:04.175Z","trace_propagation_headers":{"sentry-trace":"a3cc13f1d83d4f38a2c26daa6e9f5901-57bc20649c5c4f36","baggage":"sentry-trace_id=a3cc13f1d83d4f38a2c26daa6e9f5901,sentry-release=Unknown"},"correlation_iqemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13)2117vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.07 seconds)21182119post-build step Upload coverage to codecov: ok2120Skipping codecov: project=Mic92/nixbot attr=aarch64-linux.nixbot-gitlab